\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{fullpage}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{array}
\usepackage{color}
\usepackage{graphicx}
\usepackage{float}
\usepackage[hidelinks]{hyperref}
\usepackage{listings}
\usepackage[margin=2cm]{geometry}
\usepackage{setspace}
\usepackage{natbib}
\usepackage{proof}
\usepackage{multirow}
\usepackage{hhline}
\usepackage{wrapfig}
\usepackage [english]{babel}
\usepackage{grffile}
\usepackage{tikz}
\usepackage{pgfplots}
%\usepackage[tikz]{bclogo}
\usetikzlibrary{chains}
\usetikzlibrary{positioning}
\usetikzlibrary{arrows}
\usepackage{lscape}
\usepackage [autostyle, english = american]{csquotes}
\usepackage{enumitem}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{indentfirst}
\usepackage{hyperref}
\usepackage{pdfpages}
\usepackage{booktabs}
\usepackage{pgfgantt}
\newcommand{\tabitem}{~~\llap{\textbullet}~~}
\bibliographystyle{apsr}
\bibpunct{(}{)}{;}{a}{,}{,}
\DeclareGraphicsExtensions{.pdf,.png,.jpg}
\setlength{\tabcolsep}{.18cm}
\usepackage{fancyvrb}
\usepackage{etoc}
\usepackage{sectsty}
\sectionfont{\fontsize{14}{14}\selectfont}
\subsectionfont{\fontsize{12}{12}\selectfont}

% ==Cross Referencing Different Docs
\usepackage{xr}
\externaldocument{EfficacyRole_appendix}

\usepackage{titlesec}

\makeatletter
\@addtoreset{section}{part}
\@addtoreset{figure}{part}
\@addtoreset{table}{part}
\makeatother
\titleformat{\part}[display]
{\normalfont\LARGE\bfseries}{}{0pt}{}

\makeatletter
\renewcommand{\l@section}{\@dottedtocline{1}{1.5em}{2.6em}}
\renewcommand{\l@subsection}{\@dottedtocline{2}{4.0em}{3.6em}}
\renewcommand{\l@subsubsection}{\@dottedtocline{3}{7.4em}{4.5em}}
\makeatother

% === new commands ===
\newcommand\ud{\mathrm{d}}
\newcommand\dist{\buildrel\rm d\over\sim}
\newcommand\ind{\stackrel{\rm indep.}{\sim}}
\newcommand\iid{\stackrel{\rm i.i.d.}{\sim}}
\newcommand\logit{{\rm logit}}
\renewcommand\r{\right}
\renewcommand\l{\left}
\newcommand\Var{{\rm Var}}
\newcommand\var{{\rm var}}
\newcommand\Cov{{\rm Cov}}
\newcommand\E{\mathbb{E}}
\newcommand\V{\mathbb{V}}
\newcommand\cN{\mathcal{N}}
\newcommand\cS{\mathcal{S}}
\newcommand\cX{\mathcal{X}}
\newcommand\wX{\widetilde{X}}
\newcommand{\diag}{\mathop{\mathrm{diag}}}
\newcommand{\argmax}{\operatornamewithlimits{argmax}}
\newcommand{\argmin}{\operatornamewithlimits{argmin}}
\newcommand{\indep}{\mbox{$\perp\!\!\!\perp$}}
\def\independenT#1#2{\mathrel{\rlap{$#1#2$}\mkern2mu{#1#2}}}
\DeclareMathOperator{\sgn}{sgn}
\providecommand{\norm}[1]{\lVert#1\rVert}

\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]}
\let\endchangemargin=\endlist 

\newcolumntype{L}[1]{>{\raggedright\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{C}[1]{>{\centering\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{P}[1]{>{\raggedright\tabularxbackslash}p{#1}}

\newcounter{myWeekNum} %for gantt
\stepcounter{myWeekNum}
%
\newcommand{\myWeek}{\themyWeekNum
    \stepcounter{myWeekNum}
    \ifnum\themyWeekNum=53
         \setcounter{myWeekNum}{1}
    \else\fi
}

\newenvironment{bluetext}{\color{blue}}{\ignorespacesafterend}


\begin{document}
<<eval=TRUE, echo=FALSE, results='hide', message=FALSE>>= 

require(knitr, quietly = TRUE)

opts_chunk$set(cache = TRUE, 
               cache.path = 'cache_paper/',
               fig.path = 'figures_paper/', 
               tidy = TRUE, 
               echo = FALSE, 
               warning = FALSE, 
               message = FALSE, 
               fig.pos = 'H',
               dev = 'pdf', 
               dpi=200)

options(width = 110, digits = 2)

knit_hooks$set(inline = function(x) {
  prettyNum(x, big.mark=",")
})

@

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<<eval=TRUE, echo = FALSE, tidy=TRUE, warning=FALSE, error=FALSE, message=FALSE>>=


### load packages and functions
source("../code/VP_functions.R")

### Read in shape files
district.shp <- readOGR("../data/shapefiles/Districts.shp", verbose = FALSE)
TAN_outline <- readOGR("../data/shapefiles/TAN_outline.shp", verbose = FALSE)

### Bukoba study data
source("../code/VP_bukoba_dataclean.R")

### Kilosa study data
source("../code/VP_kilosa_dataclean.R")

### Bukoba qualitative data
source("../code/VP_bukoba_qual1_dataclean.R")
source("../code/VP_bukoba_qual2_dataclean.R")

@

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\title{{\Large \bf Self-efficacy and Citizen Engagement in Development:\\ Experimental Evidence from Tanzania}\footnote{The
data, code, and any additional materials required to replicate all analyses in this article are available at
the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: \url{https://doi.org/10.7910/DVN/SLYEUJ}. Our pre-analysis plans can be found at \url{https://osf.io/jqzxp/} for Study 1 and \url{https://osf.io/9xmjg/} for Study 2. 
This research received institutional review board (IRB) approval from MIT COUHES ($\#$1603517857R001). 
All errors and omissions are ours.
At Twaweza, we thank the many colleagues and staff members who provided feedback and contributed in the planning and implementation of the study, including Risha Chande, Glory Blasio Emmanuel, Aidan Eyakuze, Kees de Graaf, Sophia Komba, Varja Lipovsek, Benjamin Masebo, Zaida Mgalla, Kitila Mkumbo, Happiness Nkwera, Youdi Schipper, Godfrey Telli, and Richard Temu. We are grateful to Kondrada Haule, Scarion Rupia, Prosper Tarimo, and Evidence Yohana for being excellent facilitators and for their help in the refinement of the intervention and the various measurement tools; to Jasanna Britton, Amy Lieberman, and Bethany Park for their important contributions in the development of the intervention; and Mahiri Mwita for translation advice. Survey research and project implementation were carried out by Economic Development Initiatives (EDI) Limited under the supervision of Mwenge Godlaid, Marie Mallet, and Respichius Deogratias Mitti. We also thank Meghan Perdue for project administration, and Anna Weissman for qualitative coding assistance. Graeme Blair, Donghyun Danny Choi, Alexander Coppock, Benjamin Fifield, Daniel de Kadt, Kristin Michelitch, Yuki Shiraito, Lauren Young, Imai Research Group, Paluck Lab, and participants from WGAPE Abu Dhabi, MIT Political Science, Penn State Political Science, UC Berkeley Political Science, ISPS Experiments Workshop at Yale University, the International Development Research Seminar Series at LSE, APSA 2017, and ASA 2019 provided helpful comments. We thank the Editor, our Associate Editor, and anonymous reviewers. 
}
}

\author{Evan Lieberman\thanks{Department of Political Science, MIT. Email: \href{evanlieb@mit.edu}{evanlieb@mit.edu}. Address: 77 Massachusetts Avenue E53-409, Cambridge, MA 02142. Website: \href{https://evanlieberman.org/}{evanlieberman.org}} \hspace{2cm} Yang-Yang Zhou\thanks{Department of Political Science, University of British Columbia. Email: \href{yangyang.zhou@ubc.ca}{yangyang.zhou@ubc.ca}. Address: 323 C.K. Choi Building 1855 West Mall, Vancouver, BC, V6T 1Z2. Website: \href{https://www.yangyangzhou.com/}{yangyangzhou.com}}
\thanks{Authors contributed equally. Author order randomized using \url{https://randomizeauthor.shinyapps.io/shiny/}}
}

\date{\today}

%%%%%%%%%%%%%%%%% END OF PREAMBLE %%%%%%%%%%%%%%%%
\vspace{-4cm}
\maketitle

%\newpage
\vspace{-.7cm}
\begin{abstract}
\vspace{-.5cm}
\singlespacing
\noindent Recent studies of efforts to increase citizen engagement in local governance through information campaigns have reported mixed results. We consider whether low levels of self-efficacy beliefs might be limiting engagement, especially among poor citizens in poor countries. Citizens may be caught in an ``efficacy trap'' which limits their realization of better public goods provision. We describe results from a series of experimental studies conducted with over 2,200 citizens in rural Tanzania, in which we compare the effects of standard information campaigns with Validated Participation (VP), an intervention designed to socially validate citizens' participation. We implement a staged approach to experimental research, seeking to balance ethical and cost concerns about field experimentation. In our main analyses, however, we find that VP did not lead to increased levels of self-efficacy or more active citizen behaviors relative to standard informational treatments. Nonetheless, we find some promising evidence for VP in a follow-up qualitative study with teachers. We conclude by discussing lessons from this research and directions for future investigation of the possible role of self-efficacy traps in development.\\
\noindent\textbf{Keywords:} self-efficacy, development, education, experiment, Tanzania
\end{abstract}

\pagenumbering{gobble}

\newpage
\pagenumbering{arabic}
\setcounter{page}{1}

\setstretch{1.8}

\section{Introduction}

Do self-efficacy beliefs affect whether citizens engage in development-enhancing behaviors? We consider this question in the context of a popular development paradigm known as ``short-route accountability'' \citep{Bank:2003}. This model posits that citizens ought to be well-positioned to actively monitor public service inputs; and if they took individual and collective actions to apply pressure on government service providers, such as schools and health clinics, this would help to ensure acceptable levels of delivery. It offers a stark alternative to ``long-route'' or ``electoral accountability,'' in which citizen wants and needs are filtered through elections and political office-holders. While important, elections occur only every few years, and specific, localized needs can be obscured within the electoral logic, especially if elections are not competitive.

One might assume that as the beneficiaries of public services, citizens would be incentivized to take these (short-route) pro-development actions \citep{Fung:2001, Fung:2006}. Nevertheless, even in democratic countries, where citizens enjoy the political freedoms to engage in such a manner, front-line providers frequently report very low levels of citizen engagement \citep{Khemani:2016}. Social scientists have pointed to ``information asymmetries'' as the main bottleneck preventing this type of citizen engagement, particularly in low socio-economic settings \citep{Reinikka:2004, Reinikka:2005, Bjorkman:2007, Olken:2007, Banerjee:2010, Bruns:2011, Keefer:2011, Joshi:2013, Berlinski:2016, Andrabi:2017}. That is, citizens are either unaware that problems exist, or do not know what actions to take to hold officials accountable. 

Many empirical studies have investigated whether greater transparency through disseminating information to citizens would address such concerns, leading to better development outcomes \citep{Arkedis:2019}. With a few exceptions, most have found largely null or even negative effects relative to expectations. Studies carried out in India \citep{Banerjee:2010}, Chile \citep{Mizala:2013}, Kenya \citep{Lieberman:2014}, Mexico \citep{Chong:2014}, and Uganda \citep{Buntaine:2017} all tested, in one way or another, the effects of providing information to citizens, and ultimately found limited or no impact on behaviors and development outcomes.

We aim to contribute to this literature by theorizing the possibility of an \emph{efficacy trap} in citizen engagement, and we experimentally test an intervention which aims to bring individuals out of this trap. We first highlight the potential role of self-efficacy beliefs in mediating the link between information and action, identifying the importance of mastery experiences that may be in short supply among the population of interest. We report on a series of experimental studies in different regions of rural Tanzania, designed to test the effects of a simple, meeting-based intervention called \textit{Validated Participation}, which was developed in partnership with the East African civil society organization, Twaweza\footnote{\url{http://www.twaweza.org/}}. Broadly, the studies test whether self-efficacy beliefs can be augmented and sustained for increased citizen action. 

Overall, we do not find evidence that the addition of Validated Participation led to increased self-efficacy or change on parent-reported behaviors relative to subjects who only received standard information treatments. Such findings contribute to a larger body of evidence highlighting the challenges of increasing accountability through such "short-route" channels. Nonetheless, we also do not believe that the results from these experiments are fully dispositive of the theory outlined. We discuss some more encouraging findings from a small-scale, downstream study of teachers from study schools as well as more general observations that should inform future research.


\section{Theory of the Efficacy Trap}

We use the term \textit{self-efficacy} to describe a person's sense of being capable of affecting change.\footnote{\citet{Bandura:1977} originally defined self-efficacy as ``the conviction that one can successfully execute the behavior required to produce the outcomes.''} More specifically, \textit{internal efficacy} is the perception that the self has the abilities and competence to participate; while \textit{external efficacy} is the belief that institutions and authorities are responsive to the person's attempts at influence. Moreover, it is useful to distinguish between \textit{individual efficacy}, the belief that the person alone can produce change; and \textit{group efficacy}, the belief that the person, when joined by a collective, can produce change \citep{Bandura:1977, Abramson:1982, Craig:1982}. 

% THEORY IMAGE HERE
\begin{figure}[t]
    \centering
        \includegraphics[width=.7\linewidth]{figure_inputs/VPtheory.png}
    \caption{Theory of the Efficacy Trap and the role of Validated Participation. This study examines relationships depicted by shaded arrows.}
    \label{fig:theory}
\end{figure}


It stands to reason that those who lack a sense of either internal or external self-efficacy will be less likely to engage authorities and to participate in activities that they perceive are beyond their influence \citep{Smith:1989, Caprara:2009}. But what shapes self-efficacy beliefs in the first place? Prior research has demonstrated that in addition to personality traits, individuals from high-SES families or those living in high SES neighborhoods are more likely to have stronger self-efficacy beliefs \citep{Hughes:1989, Gecas:1989, Brady:1995, Boardman:2000}. Previous studies have also linked self-efficacy to political participation in both democratic \citep{Rudolph:2000,Solhaug:2006,Valentino:2009,Gottlieb:2015} and non-democratic contexts \citep{Grossman:2017,Young:2019}. Low self-efficacy citizens may not fully take advantage of service provision opportunities that demand active engagement. And yet, if such beliefs are important for citizen action, this could suggest a particularly heavy constraint on the poor, where socio-economic conditions may be difficult to change without their engagement. Indeed, this is the ``efficacy trap'' depicted in Figure \ref{fig:theory}. Because poor people with low self-efficacy may not pressure and monitor government to provide the services they need, they are ultimately less likely to capture the benefits of the very services that offer the potential to escape their lower SES status -- to escape poverty. 

Is there a way out of this efficacy trap? Also as shown in Figure \ref{fig:theory}, we propose that the central mechanism linking high SES to self-efficacy is what psychologists call \textit{mastery experiences} \citep{Bandura:1994}. People with higher education, incomes, and status enjoy more opportunities to practice participation, to be acknowledged for their efforts, and to observe some concrete changes associated with their actions. As such, they are much more likely to develop self-efficacy beliefs throughout their lives, as compared with those who lack such experiences. We consider the prospect of deliberately creating opportunities for mastery experiences that build on more universal observational and social skills that do not require high levels of education, but are also not routinely acknowledged in everyday life. This is critical, because the act of mastering a skill may not be sufficient for an individual to recognize the achievement or its value, and external acknowledgment or validation helps to cement that belief. 

We propose a specific intervention, called \textit{Validated Participation}, described in Section \ref{sec:VPIntervention}, which features discussion-based activities led by a facilitator trained to encourage and to acknowledge the contributions of participants and to provide positive feedback about their value in a collective setting. We hypothesize that this intervention can help nudge individuals out of an efficacy trap, and into a more ``virtuous circle,'' in which citizens who perceive greater self-efficacy engage more, which itself can become a mastery experience, and may ultimately lead to better services and human development outcomes. As we note in Figure \ref{fig:theory}, however, in this study, we focus our examination on just a few steps of the theory, from the exogenous introduction of the intervention as a mastery experience, to beliefs, engagement, and school-related outcomes.


\section{Citizen Engagement in Primary Education in Tanzania} 

We investigate self-efficacy beliefs and citizen participation in the context of a series of experimental studies in Tanzania, the sixth largest African country by population at 55.5 million. With a GDP per capita under \$1,000 USD, about half of the country's population lives below \$1.90 USD per day \citep{Bank:2017}. We focus on the primary education sector, a key area where citizens (generally, parents) routinely play important roles in the delivery of this public service, but in many developing country contexts, including Tanzania, such involvement is highly limited. In 2001, Tanzania introduced universal free primary education, which led to enrollment increasing from 52\% in 2000 to over 94\% of eligible children in 2008 \citep{Valente:2015}. However, the quality of public schools, teaching effort, and learning outcomes have remained poor \citep{Palmer:2007, Wedgwood:2007}. In 2015, Twaweza found that only one-third of grade 3 students could pass a grade 2 learning assessment on literacy and numeracy \citep{Twaweza:2017}. 

Additionally, both government and non-governmental actors have expressed that citizens generally do not exercise their rights to monitor teaching and school inputs or to apply pressure on officials to improve the quality of education \citep[p.8]{Mbiti:2019, Tanzania:2013}. And this sentiment was echoed in a 2015 nationally representative survey conducted by our local civil society partner, Twaweza, which found that 20\% of parents had never met with their child's teacher, 43\% had never visited the school, and 88\% had never helped out at the school within the past 12 months \citep{Twaweza:2016}. Additionally, while primary schools have school management committees (SMCs), consisting of teachers and parents, to help oversee school governance, active parental participation and collective efficacy are generally low \citep{Masue:2014, Masue:2017}. Thus, a central aim of many non-government actors has been to improve the quality of education in East Africa by increasing citizen participation and engagement. While there are many unique features that distinguish Tanzania from other countries, the core concerns are widely applicable to other contexts with underperforming social services, particularly in the education sector. 


\section{Validated Participation Intervention} 
\label{sec:VPIntervention}

% VP IMAGES HERE
\begin{figure}[t]
    \centering
    \begin{minipage}{.5\textwidth}
        \centering
        \includegraphics[width=.92\linewidth]{figure_inputs/VPimage3.JPG}
    \end{minipage}%
    \begin{minipage}{.5\textwidth}
        \centering
        \includegraphics[width=.92\linewidth]{figure_inputs/VPimage1.JPG}
    \end{minipage}
    \caption{Validated Participation intervention in progress. The facilitator is conducting a discussion about artwork with about 25 parent participants at their school (left), and later generating a list of pro-education actions that parents are volunteering and committing to take (right).}
\end{figure}

Developed in partnership with Twaweza, \textit{Validated Participation} is a novel meeting-based intervention designed to give participants opportunities for successful engagement or \textit{mastery experiences}. These were conducted in a quasi-public setting where they were validated by peers and particularly by an authority figure, namely the facilitator. In this context, the meeting group comprised about 25 adult citizens, who were parents or caretakers of children (for ease, ``parents''). The content consisted of a selection of different exercises, each of which was designed to be conceptually consistent with the theorized sources of efficacy: citizens were given equal opportunities to participate in critical discussions and decision-making, and their ideas and actions were validated by an authority figure and each other. These exercises included deliberating and voting on how to spend small grants for the school (e.g. science posters versus maps) which were almost immediately implemented, a discussion of school needs, and collectively generating a list of actions that parents could undertake to help improve their children's learning. Another unique feature of the intervention is that the facilitator displays visual images with narrative intent and encourages discussion about what participants can infer from those images. This was designed to encourage participation even among those with low literacy or initial uneasiness talking about the subject at hand. Taken together, these exercises are designed to give citizens the opportunities to practice participation through collective engagement, gaining crucial mastery experiences with positive reinforcement. The facilitator was instructed to encourage participation and to echo what individual parents said in order to signal being heard and recognized throughout the exercises. The intervention involved no new skill development -- simply eliciting and validating skills people already possessed to provide a boost of efficacy, and in so doing, to potentially facilitate more active engagement.\footnote{See SI Section \ref{sec:SIintervention} for more details on each component of the intervention.} 

We view the challenge of boosting efficacy in a relatively short period of time to be a substantial hurdle. Perceptions of efficacy are generally built up over a lifetime of experiences. Thus, one could easily imagine that a more thorough-going intervention conducted over an even longer period of time might have more enduring effects. But as a research study, it would also be more expensive to administer, and would likely suffer from greater problems of attrition. The central goal of our studies, described in the following section, was to investigate whether a short-course of intensive validation could substantively affect those perceptions and to evaluate associated behavior change.


\section{Staged Experimental Studies} 

We report on the findings of two closely-related studies, both conducted in Tanzania with our partner Twaweza: first, a school-level pilot experiment carried out in Bukoba rural district located in Kagera region in the northwest corner of the country (Study 1, Bukoba), for which we also conducted two qualitative follow-up studies six months and two years later; and second, an individual-level experimental study conducted in Kilosa district, located in Morogoro region in the center of the country (Study 2, Kilosa).\footnote{For maps of our study sites, see SI Section \ref{sec:SIdescript}.}\footnote{For a discussion on risk and ethical considerations of these studies, see SI Section \ref{sec:SIethics}.}\footnote{For a log of deviations from our pre-analysis plans, see SI Section \ref{sec:SIdeviations}.}

\subsection{Study 1: Piloting and Follow-up Interviews in Bukoba}

First, we conducted a pilot school-level study from August 2016 to January 2017 in Bukoba. We experimentally tested our intervention as a series of four weekly 90-minute meetings with a sample of \Sexpr{nrow(studyBuk)} adult caregivers\footnote{Following local usage, we refer to as `parents' throughout this paper.} across \Sexpr{nrow(Bukstudyschools)} public primary schools. Schools were randomly assigned to one of three treatment groups: (1) a pure control group in which parents were only surveyed; (2) an Information Workshop (IW) group in which parents were lectured on the importance of education and how parents could contribute; and (3) the Validated Participation (VP) group in which parents received the same informational lectures but also participated in the aforementioned exercises that were led by the facilitator.\footnote{Although a fully factorial (2 x 2) design is normally desirable to test the independent effects of both dimensions of interactive interventions (in this case, VP and IW), we have no theoretical reason to believe that Validated Participation without information would have any significant impact on parental involvement and so we do not analyze such a treatment condition.} 
Within each school, 25 eligible parents were selected by lottery balanced on gender, totaling 200 parents per treatment arm. Such a clustered design was both more expensive than an individual-level study to implement and limited statistical power given our funding constraints, but it was both more realistic in terms of how such an intervention might be delivered in practice. This design also allowed us to study outcomes over a longer period of time, because we could isolate treatment effects by school community without concern over spillovers from treated to control participants.\footnote{In SI Section \ref{sec:SIbukobaplan}, we provide additional details on this pilot study such as recruitment and block-randomization.}

Trained enumerators conducted face-to-face baseline and endline surveys in Kiswahili. We first developed an education-specific self-efficacy scale as a battery of 9 questions, with responses ranging on a scale from 1 to 5. We averaged over the questions to create an overall \textit{Efficacy Score} as well as component \textit{Internal, External, Individual, and Group Efficacy Scores}. See SI section \ref{sec:SIefficacyscale} for the full set of questions and how we constructed these indices.\footnote{We also measured the standard measures of the General Self-Efficacy Scale (battery of 10 questions, score ranges from 10 to 40) created by \citet{Jerusalem:1995}. Our self-efficacy scale is highly correlated with this standard scale. See Figure \ref{fig:EEFGEFscatter} in SI section \ref{sec:SIefficacyscale}.} We also measured parents' public efforts (speaking with teachers and monitoring their absence); private efforts (checking homework and feeding breakfast); and goals for their children (certainty of graduating secondary school). We included a behavioral component in the survey, asking parents whether they would like to leave constructive comment for the head teacher. Lastly, to examine student test scores, we conducted child learning assessments at baseline and approximately 6 months post-endline.\footnote{We used an already established assessment on basic literacy, numeracy, and reading comprehension in both English and Kiswahili developed by our partner, Twaweza. We allowed each student 15 minutes to try to complete as many questions as possible. Scores can range from 0 to 72 points. Note, however, that as indicated in our Study 1  pre-analysis plan, we did not expect to observe any effects on test scores within the context of this relatively short-term and small-sample study.} To analyze these data, we used OLS, regressing endline outcomes ($Y_{i2}$) on Validated Participation (VP) or Information Workshop (IW) treatment indicators, controlling for baseline outcome measures ($Y_{i1}$) as well as demographic covariates (${\bf x}_i$) with cluster-robust standard errors at the school level:
 
\begin{equation}
Y_{i2} = \alpha + \beta_1 VP_i + \beta_2 IW_i + \beta_3 Y_{i1} + \lambda {\bf x}_i + \epsilon_i
\end{equation}

In order to measure longer-term school-level outcomes, research assistants followed up with two waves of structured interviews with head teachers and classroom teachers of the study schools. The first was approximately six months after the conclusion of the intervention in a sample of 12 schools.\footnote{See SI Section \ref{sec:SIbukobaqual} for additional details on sampling procedures.} The second set of interviews occurred two years later in 2018 in order to study longer-term effects in all 24 study schools. We asked the teachers whether they observed changes in terms of educational outcomes (student and teacher attendance, grades, and pass rates), the performance of the specific students whose parents participated in the study, and the responsiveness of local government officials to the school's needs. We also asked them to tell us about levels of parental engagement at the school (taking any notable actions at school, interacting with teachers regularly, participating at meetings and whether those who speak at the meetings are mostly male elders or more diverse, making material contributions to the school, and helping with schoolwork). Although such actors might ordinarily be induced to over-report positive outcomes associated with the study, these tendencies would bias against detecting any effect. Furthermore, all of the schools in the Bukoba study were considered ``study'' sites, thus the teachers were unaware of treatment assignment status. Thus, we would expect to see any social desirability or related biases to be similar in responses across all three treatment arms. 
We then employed multiple research assistants to independently code these interview transcripts.
To analyze these interview data, we use OLS regressing outcomes ($Y_{i}$) on Validated Participation (VP) or Information Workshop (IW) treatment indicators, with respondent-type fixed effects (head or classroom teacher, indexed by $t$) and cluster-robust standard errors at the school level:

\begin{equation}
Y_{i} = \alpha_t + \beta_1 VP_i + \beta_2 IW_i + \epsilon_{i}
\end{equation}


\subsection{Study 2: Individual-Level Random Assignment in Kilosa}

In 2018, we carried out Study 2, which examined the effects of a single 90-minute meeting with a sample of \Sexpr{nrow(studyKil)} parents across \Sexpr{nrow(KilSchoolMain)-1} public primary schools.\footnote{This study was designed to detect the effect of treatment on efficacy beliefs with a power of .8, based on results from piloting.}\footnote{Although in our original pre-analysis plan, we had planned for a fully-powered school-level study, we were unable to attain sufficient funding, thus we proceeded with the individual-level Study 2. Although the intervention itself is low-cost, the implementation of the study is extremely expensive.} Parents were recruited to schools and upon arrival, assigned to one of three treatment arms: (1) a pure control group in which parents were only surveyed; (2) an Information Workshop (IW) group in which parents watched an informational video on education; and (3) the Validated Participation (VP) group in which parents watched the same video but also participated in the aforementioned exercises that were led by the facilitator.

Approximately 540 parents were randomly assigned to each treatment arm, and each group's activities were held in separate classrooms to guard against spillover effects -- specifically, the possibility that those who received different treatments would discuss their experiences and influence one another's attitudes and behaviors, contaminating our ability to disentangle the effects of the different treatment conditions. Enumerators conducted a face-to-face survey with parents in Kiswahili immediately after the meeting. In this context, any experimental intervention that involves a potentially foreign-funded NGO may elicit responses on surveys and in practice that correspond to what citizens believe the NGO wants to ``see'' or ``hear,'' with the hopes that such compliance will be rewarded with financial or other forms of support. Recognizing this, both of our studies hold such pressures constant across treatment and control groups, which ought to weigh against being able to detect treatment effects in an experimental setting.\footnote{We assume here that treatment does not interact with social desirability bias.} 
Additionally, we asked about willingness to take certain actions related to government accountability (supporting a pro-education political candidate, raising a public goods issue at a community meeting, raising that issue with a government official) and feelings of being respected by others (peers, authorities, the SMC).

As the design of Study 2 was cross-sectional, we use OLS regressing outcomes ($Y_{i}$) on Validated Participation (VP) or Information Workshop (IW) treatment indicators, controlling for demographic covariates for covariate adjustment (${\bf x}_i$), with school fixed effects:
\begin{equation}
Y_{i} = \alpha_s + \beta_1 VP_i + \beta_2 IW_i + \lambda {\bf x}_i + \epsilon_i
\end{equation}


\section{Association between Efficacy and Active Citizenship}
 
Before turning to the experimental results, we first explore the theorized link between efficacy and and pro-development behaviors through observational analyses of naturally-occurring variation in efficacy beliefs in both study settings. Figure \ref{fig:EEFplot} shows the positive associations between self-efficacy and self-reported parental behaviors as well as child assessment scores at baseline, controlling for all individual and household socio-demographic covariates.
\footnote{Figure \ref{fig:EEFplot2} in SI Section \ref{sec:SIEEF2} shows that our educational self-efficacy scale is more strongly associated with pro-development outcomes compared to the general self-efficacy scale by \citet{Jerusalem:1995}.} 


<<EEFfit_Buk, eval = TRUE, echo = FALSE, tidy=TRUE, warning = FALSE, message = FALSE, results='asis'>>=

## Bukoba study: Fit regression models with Baseline Education Efficacy score, Household Wealth, and Parent's education level as PREDICTORS to behavioral outcomes

# Outcome: Knows teacher's name
EEFBfit.CCK1_B_num <- lm_robust(CCK1_B_num ~ EEFscore_B + # EEF score
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFBfit.CCK1_B_num <- lm_robust(CCK1_B_num ~ GEFscore_B + # GEF score
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Econfit.CCK1_B_num <- lm_robust(CCK1_B_num ~ econindexSUM_B_rescaled + # HH wealth rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + disabilities_B_bi +
             CH1_B + BC5_B_num + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Edufit.CCK1_B_num <- lm_robust(CCK1_B_num ~ BC5_B_num_rescaled + # Education rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)
                          
# Outcome: Believes child will complete secondary school
EEFBfit.EQ5_B_bi <- lm_robust(EQ5_B_bi ~ EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFBfit.EQ5_B_bi <- lm_robust(EQ5_B_bi ~ GEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Econfit.EQ5_B_bi <- lm_robust(EQ5_B_bi ~ econindexSUM_B_rescaled + # HH wealth rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + disabilities_B_bi +
             CH1_B + BC5_B_num + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Edufit.EQ5_B_bi <- lm_robust(EQ5_B_bi ~ BC5_B_num_rescaled + # Education rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

# Outcome: Asked about teacher absence last week?
EEFBfit.EQ1_B_num <- lm_robust(EQ1_B_num ~ EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFBfit.EQ1_B_num <- lm_robust(EQ1_B_num ~ GEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Econfit.EQ1_B_num <- lm_robust(EQ1_B_num ~ econindexSUM_B_rescaled + # HH wealth rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + disabilities_B_bi +
             CH1_B + BC5_B_num + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Edufit.EQ1_B_num <- lm_robust(EQ1_B_num ~ BC5_B_num_rescaled + # Education rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

# Outcome: Baseline child test scores 
EEFBfit.KFTotal_B <- lm_robust(KFTotal_B ~ EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFBfit.KFTotal_B <- lm_robust(KFTotal_B ~ GEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Econfit.KFTotal_B <- lm_robust(KFTotal_B ~ econindexSUM_B_rescaled + # HH wealth rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + disabilities_B_bi +
             CH1_B + BC5_B_num + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Edufit.KFTotal_B <- lm_robust(KFTotal_B ~ BC5_B_num_rescaled + # Education rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

# Outcome: Looked at HW last week?
EEFBfit.PP2_B_num <- lm_robust(PP2_B_num ~ EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFBfit.PP2_B_num <- lm_robust(PP2_B_num ~ GEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Econfit.PP2_B_num <- lm_robust(PP2_B_num ~ econindexSUM_B_rescaled + # HH wealth rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + disabilities_B_bi +
             CH1_B + BC5_B_num + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Edufit.PP2_B_num <- lm_robust(PP2_B_num ~ BC5_B_num_rescaled + # Education rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

# Outcome: Child had breakfast today?
EEFBfit.CB1_B_num <- lm_robust(CB1_B_num ~ EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFBfit.CB1_B_num <- lm_robust(CB1_B_num ~ GEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Econfit.CB1_B_num <- lm_robust(CB1_B_num ~ econindexSUM_B_rescaled + # HH wealth rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + disabilities_B_bi +
             CH1_B + BC5_B_num + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

Edufit.CB1_B_num <- lm_robust(CB1_B_num ~ BC5_B_num_rescaled + # Education rescaled
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + EEFscore_B + AgeBin,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

## Bukoba Study Generate predicted probs 
# Knows teacher's name?
EEFBpred.CCK1_B_num <- fitpredEEF(EEFBfit.CCK1_B_num, 
                                  data = studyBuk, 
                                  var = "EEFscore_B") 
EEFBpred.CCK1_B_num <- as.data.frame(lapply(EEFBpred.CCK1_B_num, function(x) as.numeric(as.character(x))))
EEFBpred.CCK1_B_num$var <- "EEFB"

GEFBpred.CCK1_B_num <- fitpredEEF(GEFBfit.CCK1_B_num, 
                                  data = studyBuk, 
                                  var = "GEFscore_B") 
GEFBpred.CCK1_B_num <- as.data.frame(lapply(GEFBpred.CCK1_B_num, function(x) as.numeric(as.character(x))))
GEFBpred.CCK1_B_num$var <- "GEFB"
  
Econpred.CCK1_B_num <- fitpredEEF(Econfit.CCK1_B_num, 
                                  data = studyBuk, 
                                  var = "econindexSUM_B_rescaled") 
Econpred.CCK1_B_num <- as.data.frame(lapply(Econpred.CCK1_B_num, function(x) as.numeric(as.character(x))))
Econpred.CCK1_B_num$var <- "Econ"

Edupred.CCK1_B_num <- fitpredEEF(Edufit.CCK1_B_num, 
                                  data = studyBuk, 
                                  var = "BC5_B_num_rescaled") 
Edupred.CCK1_B_num <- as.data.frame(lapply(Edupred.CCK1_B_num, function(x) as.numeric(as.character(x))))
Edupred.CCK1_B_num$var <- "Edu"

# Certain child will complete secondary school
EEFBpred.EQ5_B_bi <- fitpredEEF(EEFBfit.EQ5_B_bi, 
                                 data = studyBuk, 
                                 var = "EEFscore_B") 
EEFBpred.EQ5_B_bi <- as.data.frame(lapply(EEFBpred.EQ5_B_bi, function(x) as.numeric(as.character(x))))
EEFBpred.EQ5_B_bi$var <- "EEFB"

GEFBpred.EQ5_B_bi <- fitpredEEF(GEFBfit.EQ5_B_bi, 
                                 data = studyBuk, 
                                 var = "GEFscore_B") 
GEFBpred.EQ5_B_bi <- as.data.frame(lapply(GEFBpred.EQ5_B_bi, function(x) as.numeric(as.character(x))))
GEFBpred.EQ5_B_bi$var <- "GEFB"

Econpred.EQ5_B_bi <- fitpredEEF(Econfit.EQ5_B_bi, 
                                  data = studyBuk, 
                                  var = "econindexSUM_B_rescaled") 
Econpred.EQ5_B_bi <- as.data.frame(lapply(Econpred.EQ5_B_bi, function(x) as.numeric(as.character(x))))
Econpred.EQ5_B_bi$var <- "Econ"

Edupred.EQ5_B_bi <- fitpredEEF(Edufit.EQ5_B_bi, 
                                  data = studyBuk, 
                                  var = "BC5_B_num_rescaled") 
Edupred.EQ5_B_bi <- as.data.frame(lapply(Edupred.EQ5_B_bi, function(x) as.numeric(as.character(x))))
Edupred.EQ5_B_bi$var <- "Edu"

# Asked about teacher absence last week?
EEFBpred.EQ1_B_num <- fitpredEEF(EEFBfit.EQ1_B_num, 
                                 data = studyBuk, 
                                 var = "EEFscore_B") 
EEFBpred.EQ1_B_num <- as.data.frame(lapply(EEFBpred.EQ1_B_num, function(x) as.numeric(as.character(x))))
EEFBpred.EQ1_B_num$var <- "EEFB"

GEFBpred.EQ1_B_num <- fitpredEEF(GEFBfit.EQ1_B_num, 
                                 data = studyBuk, 
                                 var = "GEFscore_B") 
GEFBpred.EQ1_B_num <- as.data.frame(lapply(GEFBpred.EQ1_B_num, function(x) as.numeric(as.character(x))))
GEFBpred.EQ1_B_num$var <- "GEFB"

Econpred.EQ1_B_num <- fitpredEEF(Econfit.EQ1_B_num, 
                                  data = studyBuk, 
                                  var = "econindexSUM_B_rescaled") 
Econpred.EQ1_B_num <- as.data.frame(lapply(Econpred.EQ1_B_num, function(x) as.numeric(as.character(x))))
Econpred.EQ1_B_num$var <- "Econ"

Edupred.EQ1_B_num <- fitpredEEF(Edufit.EQ1_B_num, 
                                  data = studyBuk, 
                                  var = "BC5_B_num_rescaled") 
Edupred.EQ1_B_num <- as.data.frame(lapply(Edupred.EQ1_B_num, function(x) as.numeric(as.character(x))))
Edupred.EQ1_B_num$var <- "Edu"

# Baseline test scores
EEFBpred.KFTotal_B <- fitpredEEF(EEFBfit.KFTotal_B, 
                                 data = studyBuk, 
                                 var = "EEFscore_B") 
EEFBpred.KFTotal_B <- as.data.frame(lapply(EEFBpred.KFTotal_B, function(x) as.numeric(as.character(x))))
EEFBpred.KFTotal_B$var <- "EEFB"

GEFBpred.KFTotal_B <- fitpredEEF(GEFBfit.KFTotal_B, 
                                 data = studyBuk, 
                                 var = "GEFscore_B") 
GEFBpred.KFTotal_B <- as.data.frame(lapply(GEFBpred.KFTotal_B, function(x) as.numeric(as.character(x))))
GEFBpred.KFTotal_B$var <- "GEFB"

Econpred.KFTotal_B <- fitpredEEF(Econfit.KFTotal_B, 
                                  data = studyBuk, 
                                  var = "econindexSUM_B_rescaled") 
Econpred.KFTotal_B <- as.data.frame(lapply(Econpred.KFTotal_B, function(x) as.numeric(as.character(x))))
Econpred.KFTotal_B$var <- "Econ"

Edupred.KFTotal_B <- fitpredEEF(Edufit.KFTotal_B, 
                                  data = studyBuk, 
                                  var = "BC5_B_num_rescaled") 
Edupred.KFTotal_B <- as.data.frame(lapply(Edupred.KFTotal_B, function(x) as.numeric(as.character(x))))
Edupred.KFTotal_B$var <- "Edu"

# Looked at HW last week?
EEFBpred.PP2_B_num <- fitpredEEF(EEFBfit.PP2_B_num, 
                                 data = studyBuk, 
                                 var = "EEFscore_B") 
EEFBpred.PP2_B_num <- as.data.frame(lapply(EEFBpred.PP2_B_num, function(x) as.numeric(as.character(x))))
EEFBpred.PP2_B_num$var <- "EEFB"

GEFBpred.PP2_B_num <- fitpredEEF(GEFBfit.PP2_B_num, 
                                 data = studyBuk, 
                                 var = "GEFscore_B") 
GEFBpred.PP2_B_num <- as.data.frame(lapply(GEFBpred.PP2_B_num, function(x) as.numeric(as.character(x))))
GEFBpred.PP2_B_num$var <- "GEFB"

Econpred.PP2_B_num <- fitpredEEF(Econfit.PP2_B_num, 
                                  data = studyBuk, 
                                  var = "econindexSUM_B_rescaled") 
Econpred.PP2_B_num <- as.data.frame(lapply(Econpred.PP2_B_num, function(x) as.numeric(as.character(x))))
Econpred.PP2_B_num$var <- "Econ"

Edupred.PP2_B_num <- fitpredEEF(Edufit.PP2_B_num, 
                                  data = studyBuk, 
                                  var = "BC5_B_num_rescaled") 
Edupred.PP2_B_num <- as.data.frame(lapply(Edupred.PP2_B_num, function(x) as.numeric(as.character(x))))
Edupred.PP2_B_num$var <- "Edu"

# Child had breakfast today?
EEFBpred.CB1_B_num <- fitpredEEF(EEFBfit.CB1_B_num, 
                                 data = studyBuk, 
                                 var = "EEFscore_B") 
EEFBpred.CB1_B_num <- as.data.frame(lapply(EEFBpred.CB1_B_num, function(x) as.numeric(as.character(x))))
EEFBpred.CB1_B_num$var <- "EEFB"

GEFBpred.CB1_B_num <- fitpredEEF(GEFBfit.CB1_B_num, 
                                 data = studyBuk, 
                                 var = "GEFscore_B") 
GEFBpred.CB1_B_num <- as.data.frame(lapply(GEFBpred.CB1_B_num, function(x) as.numeric(as.character(x))))
GEFBpred.CB1_B_num$var <- "GEFB"

Econpred.CB1_B_num <- fitpredEEF(Econfit.CB1_B_num, 
                                  data = studyBuk, 
                                  var = "econindexSUM_B_rescaled") 
Econpred.CB1_B_num <- as.data.frame(lapply(Econpred.CB1_B_num, function(x) as.numeric(as.character(x))))
Econpred.CB1_B_num$var <- "Econ"

Edupred.CB1_B_num <- fitpredEEF(Edufit.CB1_B_num, 
                                  data = studyBuk, 
                                  var = "BC5_B_num_rescaled") 
Edupred.CB1_B_num <- as.data.frame(lapply(Edupred.CB1_B_num, function(x) as.numeric(as.character(x))))
Edupred.CB1_B_num$var <- "Edu"

## Save fit objects and predicted probability datasets as Rdata
dir.create("../output/")
save(EEFBpred.CCK1_B_num, EEFBfit.CCK1_B_num,
     EEFBpred.EQ5_B_bi, EEFBpred.EQ5_B_bi,
     EEFBpred.EQ1_B_num, EEFBfit.EQ1_B_num,
     EEFBpred.KFTotal_B, EEFBfit.KFTotal_B,
     EEFBpred.PP2_B_num, EEFBfit.PP2_B_num,
     EEFBpred.CB1_B_num, EEFBfit.CB1_B_num,
     file = "../output/BukEEFbasefit.Rdata")

save(GEFBpred.CCK1_B_num, GEFBfit.CCK1_B_num,
     GEFBpred.EQ5_B_bi, GEFBpred.EQ5_B_bi,
     GEFBpred.EQ1_B_num, GEFBfit.EQ1_B_num,
     GEFBpred.KFTotal_B, GEFBfit.KFTotal_B,
     GEFBpred.PP2_B_num, GEFBfit.PP2_B_num,
     GEFBpred.CB1_B_num, GEFBfit.CB1_B_num,
     file = "../output/BukGEFbasefit.Rdata")

save(Econpred.CCK1_B_num, Econfit.CCK1_B_num,
     Econpred.EQ5_B_bi, Econpred.EQ5_B_bi,
     Econpred.EQ1_B_num, Econfit.EQ1_B_num,
     Econpred.KFTotal_B, Econfit.KFTotal_B,
     Econpred.PP2_B_num, Econfit.PP2_B_num,
     Econpred.CB1_B_num, Econfit.CB1_B_num,
     file = "../output/BukEconfit.Rdata")

save(Edupred.CCK1_B_num, Edufit.CCK1_B_num,
     Edupred.EQ5_B_bi, Edupred.EQ5_B_bi,
     Edupred.EQ1_B_num, Edufit.EQ1_B_num,
     Edupred.KFTotal_B, Edufit.KFTotal_B,
     Edupred.PP2_B_num, Edufit.PP2_B_num,
     Edupred.CB1_B_num, Edufit.CB1_B_num,
     file = "../output/BukEdufit.Rdata")

@

<<EEFfit_Kil, eval = TRUE, echo = FALSE, tidy=TRUE, warning = FALSE, message = FALSE, results='asis'>>=

## Kilosa study: Fit regression models with Education Efficacy score, Household Wealth, and Parent's education level as PREDICTORS to behavioral outcomes

# Outcome: Respected by Parents
EEFfit.RespectedbyParents <- lm_robust(RespectedbyParents ~ EEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

GEFfit.RespectedbyParents <- lm_robust(RespectedbyParents ~ GEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Econfit.RespectedbyParents <- lm_robust(RespectedbyParents ~ econindexSUM_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + EEFscore +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Edufit.RespectedbyParents <- lm_robust(RespectedbyParents ~ Education_num_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               EEFscore,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

# Outcome: Respected by Authorities
EEFfit.RespectedbyAuth <- lm_robust(RespectedbyAuth ~ EEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

GEFfit.RespectedbyAuth <- lm_robust(RespectedbyAuth ~ GEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Econfit.RespectedbyAuth <- lm_robust(RespectedbyAuth ~ econindexSUM_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + EEFscore +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Edufit.RespectedbyAuth <- lm_robust(RespectedbyAuth ~ Education_num_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               EEFscore,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

# Outcome: SC Ideas Respected
EEFfit.RespectedIdeas <- lm_robust(RespectedIdeas ~ EEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

GEFfit.RespectedIdeas <- lm_robust(RespectedIdeas ~ GEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Econfit.RespectedIdeas <- lm_robust(RespectedIdeas ~ econindexSUM_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + EEFscore +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Edufit.RespectedIdeas <- lm_robust(RespectedIdeas ~ Education_num_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               EEFscore,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

# Outcome: Public Goods Mtg
EEFfit.PubGoodMtg <- lm_robust(PubGoodMtg ~ EEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

GEFfit.PubGoodMtg <- lm_robust(PubGoodMtg ~ GEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Econfit.PubGoodMtg <- lm_robust(PubGoodMtg ~ econindexSUM_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + EEFscore +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Edufit.PubGoodMtg <- lm_robust(PubGoodMtg ~ Education_num_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               EEFscore,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

# Outcome: Public Goods Direct
EEFfit.PubGoodDirect <- lm_robust(PubGoodDirect ~ EEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

GEFfit.PubGoodDirect <- lm_robust(PubGoodDirect ~ GEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Econfit.PubGoodDirect <- lm_robust(PubGoodDirect ~ econindexSUM_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + EEFscore +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Edufit.PubGoodDirect <- lm_robust(PubGoodDirect ~ Education_num_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               EEFscore,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

# Outcome: Support Pro-Education Candidate
EEFfit.Candidate <- lm_robust(Candidate ~ EEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

GEFfit.Candidate <- lm_robust(Candidate ~ GEFscore + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Econfit.Candidate <- lm_robust(Candidate ~ econindexSUM_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + EEFscore +
               Education_num,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

Edufit.Candidate <- lm_robust(Candidate ~ Education_num_rescaled + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               EEFscore,
               fixed_effects = ~SchoolID,
               alpha = .1, 
               clusters = SchoolID,
               data = SOKil)

## Kilosa Study Generate predicted probs 
# Respected by parents
EEFpred.RespectedbyParents <- fitpredEEF(EEFfit.RespectedbyParents, 
                                          data = SOKil, 
                                          var = "EEFscore") 
EEFpred.RespectedbyParents <- as.data.frame(lapply(EEFpred.RespectedbyParents, 
                                                   function(x) as.numeric(as.character(x))))
EEFpred.RespectedbyParents$var <- "EEF"

GEFpred.RespectedbyParents <- fitpredEEF(GEFfit.RespectedbyParents, 
                                          data = SOKil, 
                                          var = "GEFscore") 
GEFpred.RespectedbyParents <- as.data.frame(lapply(GEFpred.RespectedbyParents, 
                                                   function(x) as.numeric(as.character(x))))
GEFpred.RespectedbyParents$var <- "GEF"

Econpred.RespectedbyParents <- fitpredEEF(Econfit.RespectedbyParents, 
                                          data = SOKil, 
                                          var = "econindexSUM_rescaled") 
Econpred.RespectedbyParents <- as.data.frame(lapply(Econpred.RespectedbyParents, 
                                                    function(x) as.numeric(as.character(x))))
Econpred.RespectedbyParents$var <- "Econ"

Edupred.RespectedbyParents <- fitpredEEF(Edufit.RespectedbyParents, 
                                          data = SOKil, 
                                          var = "Education_num_rescaled") 
Edupred.RespectedbyParents <- as.data.frame(lapply(Edupred.RespectedbyParents, 
                                                   function(x) as.numeric(as.character(x))))
Edupred.RespectedbyParents$var <- "Edu"

# Respected by authorities
EEFpred.RespectedbyAuth <- fitpredEEF(EEFfit.RespectedbyAuth, 
                                          data = SOKil, 
                                          var = "EEFscore") 
EEFpred.RespectedbyAuth <- as.data.frame(lapply(EEFpred.RespectedbyAuth, 
                                                function(x) as.numeric(as.character(x))))
EEFpred.RespectedbyAuth$var <- "EEF"

GEFpred.RespectedbyAuth <- fitpredEEF(GEFfit.RespectedbyAuth, 
                                          data = SOKil, 
                                          var = "GEFscore") 
GEFpred.RespectedbyAuth <- as.data.frame(lapply(GEFpred.RespectedbyAuth, 
                                                function(x) as.numeric(as.character(x))))
GEFpred.RespectedbyAuth$var <- "GEF"

Econpred.RespectedbyAuth <- fitpredEEF(Econfit.RespectedbyAuth, 
                                          data = SOKil, 
                                          var = "econindexSUM_rescaled") 
Econpred.RespectedbyAuth <- as.data.frame(lapply(Econpred.RespectedbyAuth, 
                                                 function(x) as.numeric(as.character(x))))
Econpred.RespectedbyAuth$var <- "Econ"

Edupred.RespectedbyAuth <- fitpredEEF(Edufit.RespectedbyAuth, 
                                          data = SOKil, 
                                          var = "Education_num_rescaled") 
Edupred.RespectedbyAuth <- as.data.frame(lapply(Edupred.RespectedbyAuth, 
                                                function(x) as.numeric(as.character(x))))
Edupred.RespectedbyAuth$var <- "Edu"

# Ideas respected
EEFpred.RespectedIdeas<- fitpredEEF(EEFfit.RespectedIdeas, 
                                          data = SOKil, 
                                          var = "EEFscore") 
EEFpred.RespectedIdeas <- as.data.frame(lapply(EEFpred.RespectedIdeas, 
                                               function(x) as.numeric(as.character(x))))
EEFpred.RespectedIdeas$var <- "EEF"

GEFpred.RespectedIdeas<- fitpredEEF(GEFfit.RespectedIdeas, 
                                          data = SOKil, 
                                          var = "GEFscore") 
GEFpred.RespectedIdeas <- as.data.frame(lapply(GEFpred.RespectedIdeas, 
                                               function(x) as.numeric(as.character(x))))
GEFpred.RespectedIdeas$var <- "GEF"

Econpred.RespectedIdeas <- fitpredEEF(Econfit.RespectedIdeas, 
                                          data = SOKil, 
                                          var = "econindexSUM_rescaled") 
Econpred.RespectedIdeas <- as.data.frame(lapply(Econpred.RespectedIdeas, 
                                                function(x) as.numeric(as.character(x))))
Econpred.RespectedIdeas$var <- "Econ"

Edupred.RespectedIdeas <- fitpredEEF(Edufit.RespectedIdeas, 
                                          data = SOKil, 
                                          var = "Education_num_rescaled") 
Edupred.RespectedIdeas <- as.data.frame(lapply(Edupred.RespectedIdeas, 
                                               function(x) as.numeric(as.character(x))))
Edupred.RespectedIdeas$var <- "Edu"

#Public Goods Mtg
EEFpred.PubGoodMtg <- fitpredEEF(EEFfit.PubGoodMtg, 
                                          data = SOKil, 
                                          var = "EEFscore") 
EEFpred.PubGoodMtg <- as.data.frame(lapply(EEFpred.PubGoodMtg, 
                                           function(x) as.numeric(as.character(x))))
EEFpred.PubGoodMtg$var <- "EEF"

GEFpred.PubGoodMtg <- fitpredEEF(GEFfit.PubGoodMtg, 
                                          data = SOKil, 
                                          var = "GEFscore") 
GEFpred.PubGoodMtg <- as.data.frame(lapply(GEFpred.PubGoodMtg, 
                                           function(x) as.numeric(as.character(x))))
GEFpred.PubGoodMtg$var <- "GEF"

Econpred.PubGoodMtg <- fitpredEEF(Econfit.PubGoodMtg, 
                                          data = SOKil, 
                                          var = "econindexSUM_rescaled") 
Econpred.PubGoodMtg <- as.data.frame(lapply(Econpred.PubGoodMtg, 
                                            function(x) as.numeric(as.character(x))))
Econpred.PubGoodMtg$var <- "Econ"

Edupred.PubGoodMtg <- fitpredEEF(Edufit.PubGoodMtg, 
                                          data = SOKil, 
                                          var = "Education_num_rescaled") 
Edupred.PubGoodMtg <- as.data.frame(lapply(Edupred.PubGoodMtg, 
                                           function(x) as.numeric(as.character(x))))
Edupred.PubGoodMtg$var <- "Edu"

# Public Goods Direct
EEFpred.PubGoodDirect <- fitpredEEF(EEFfit.PubGoodDirect, 
                                          data = SOKil, 
                                          var = "EEFscore") 
EEFpred.PubGoodDirect <- as.data.frame(lapply(EEFpred.PubGoodDirect, 
                                              function(x) as.numeric(as.character(x))))
EEFpred.PubGoodDirect$var <- "EEF"

GEFpred.PubGoodDirect <- fitpredEEF(GEFfit.PubGoodDirect, 
                                          data = SOKil, 
                                          var = "GEFscore") 
GEFpred.PubGoodDirect <- as.data.frame(lapply(GEFpred.PubGoodDirect, 
                                              function(x) as.numeric(as.character(x))))
GEFpred.PubGoodDirect$var <- "GEF"

Econpred.PubGoodDirect <- fitpredEEF(Econfit.PubGoodDirect, 
                                          data = SOKil, 
                                          var = "econindexSUM_rescaled") 
Econpred.PubGoodDirect <- as.data.frame(lapply(Econpred.PubGoodDirect, 
                                               function(x) as.numeric(as.character(x))))
Econpred.PubGoodDirect$var <- "Econ"

Edupred.PubGoodDirect <- fitpredEEF(Edufit.PubGoodDirect,  
                                          data = SOKil, 
                                          var = "Education_num_rescaled") 
Edupred.PubGoodDirect <- as.data.frame(lapply(Edupred.PubGoodDirect, 
                                              function(x) as.numeric(as.character(x))))
Edupred.PubGoodDirect$var <- "Edu"

# Support Pro-Education Candidate
EEFpred.Candidate <- fitpredEEF(EEFfit.Candidate, 
                                          data = SOKil, 
                                          var = "EEFscore") 
EEFpred.Candidate <- as.data.frame(lapply(EEFpred.Candidate, 
                                          function(x) as.numeric(as.character(x))))
EEFpred.Candidate$var <- "EEF"

GEFpred.Candidate <- fitpredEEF(GEFfit.Candidate, 
                                          data = SOKil, 
                                          var = "GEFscore") 
GEFpred.Candidate <- as.data.frame(lapply(GEFpred.Candidate, 
                                          function(x) as.numeric(as.character(x))))
GEFpred.Candidate$var <- "GEF"

Econpred.Candidate <- fitpredEEF(Econfit.Candidate, 
                                          data = SOKil, 
                                          var = "econindexSUM_rescaled") 
Econpred.Candidate <- as.data.frame(lapply(Econpred.Candidate, 
                                           function(x) as.numeric(as.character(x))))
Econpred.Candidate$var <- "Econ"

Edupred.Candidate <- fitpredEEF(Edufit.Candidate,  
                                          data = SOKil, 
                                          var = "Education_num_rescaled") 
Edupred.Candidate <- as.data.frame(lapply(Edupred.Candidate, 
                                          function(x) as.numeric(as.character(x))))
Edupred.Candidate$var <- "Edu"

#Save fit objects and predicted probability datasets as Rdata
save(EEFpred.RespectedbyParents, EEFfit.RespectedbyParents,
     EEFpred.RespectedbyAuth, EEFfit.RespectedbyAuth,
     EEFpred.RespectedIdeas, EEFfit.RespectedIdeas,
     EEFpred.PubGoodMtg, EEFfit.PubGoodMtg,
     EEFpred.PubGoodDirect, EEFfit.PubGoodDirect,
     EEFpred.Candidate, EEFfit.Candidate,
     file = "../output/KilEEFfit.Rdata")

save(GEFpred.RespectedbyParents, GEFfit.RespectedbyParents,
     GEFpred.RespectedbyAuth, GEFfit.RespectedbyAuth,
     GEFpred.RespectedIdeas, GEFfit.RespectedIdeas,
     GEFpred.PubGoodMtg, GEFfit.PubGoodMtg,
     GEFpred.PubGoodDirect, GEFfit.PubGoodDirect,
     GEFpred.Candidate, GEFfit.Candidate,
     file = "../output/KilGEFfit.Rdata")

save(Econpred.RespectedbyParents, Econfit.RespectedbyParents,
     Econpred.RespectedbyAuth, Econfit.RespectedbyAuth,
     Econpred.RespectedIdeas, Econfit.RespectedIdeas,
     Econpred.PubGoodMtg, Econfit.PubGoodMtg,
     Econpred.PubGoodDirect, Econfit.PubGoodDirect,
     Econpred.Candidate, Econfit.Candidate,
     file = "../output/KilEconfit.Rdata")

save(Edupred.RespectedbyParents, Edufit.RespectedbyParents,
     Edupred.RespectedbyAuth, Edufit.RespectedbyAuth,
     Edupred.RespectedIdeas, Edufit.RespectedIdeas,
     Edupred.PubGoodMtg, Edufit.PubGoodMtg,
     Edupred.PubGoodDirect, Edufit.PubGoodDirect,
     Edupred.Candidate, Edufit.Candidate,
     file = "../output/KilEdufit.Rdata")

@

<<EEFplot, eval=TRUE, echo = FALSE, tidy=TRUE, fig.width = 10, fig.height = 10, out.width= ".85\\linewidth", fig.pos = 'H', fig.align='center', warning=FALSE, message=FALSE, fig.cap="Positive linear relationships between our educational self-efficacy score (x-axis) and citizen participation / pro-development outcomes (y-axis) from both studies. From Study 1 Bukoba (rows 1-2), these plots show the predicted probabilities for student test scores and parental outcomes at baseline. From Study 2 Kilosa (row 3), these plots show the predicted probabilities for pro-education political behaviors for the SO (control) parents (with 95$\\%$ CIs). OLS models, controlling for demographic covariates, school cluster robust standard errors.">>=

# Load predicted probability data from Bukoba and Kilosa studies
# setwd("Paper_Inputs")
# 
# load("BukEEFbasefit.Rdata")
# load("BukGEFbasefit.Rdata")
# load("BukEconfit.Rdata")
# load("BukEdufit.Rdata")
# 
# load("KilEEFfit.Rdata")
# load("KilGEFfit.Rdata")
# load("KilEconfit.Rdata")
# load("KilEdufit.Rdata")

## Bukoba study plots
# Child's test scores 
EEFBplot.KFTotal_B <- ggplot(EEFBpred.KFTotal_B, 
                            aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(0, 30) +
  ylab("Predicted Test score") +
  xlab("Pre-Treatment Score") +
  ggtitle("Child's test scores (/72 pts)") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Knows teacher's name
EEFBplot.CCK1_B_num <- ggplot(EEFBpred.CCK1_B_num, 
                              aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(0, 1) +
  ylab("Predicted Probability") +
  xlab("Pre-Treatment Score") +
  ggtitle("Knows teacher's name") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Certain graduate secondary school
EEFBplot.EQ5_B_bi <- ggplot(EEFBpred.EQ5_B_bi,
                            aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(-.27, 1) +
  ylab("Predicted Probability") +
  xlab("Pre-Treatment Score") +
  ggtitle("Believes child will graduate secondary") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 11),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Asks about teacher absence
EEFBplot.EQ1_B_num <- ggplot(EEFBpred.EQ1_B_num, 
                            aes(x = EEFscore, y = est)) +  
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(-.02, 1) +
  ylab("Predicted Probability") +
  xlab("Pre-Treatment Score") +
  ggtitle("Asks about teacher absence") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Looked at HW last week
EEFBplot.PP2_B_num <- ggplot(EEFBpred.PP2_B_num, 
                            aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(0, 1) +
  ylab("Predicted Probability") +
  xlab("Pre-Treatment Score") +
  ggtitle("Looked at HW last week") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Gave child breakfast today
EEFBplot.CB1_B_num <- ggplot(EEFBpred.CB1_B_num, 
                            aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(0, 1) +
  ylab("Predicted Probability") +
  xlab("Pre-Treatment Score") +
  ggtitle("Gave child breakfast today") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  


## Kilosa study plots
# Respected by other parents
EEFplot.RespectedbyParents <- ggplot(EEFpred.RespectedbyParents, 
                                     aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(1, 5) +
  ylab("Predicted Response on 1 to 5 scale") +
  xlab("Control Group Score") +
  ggtitle("Respected by other parents") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Raise public goods issue at mtg
EEFplot.PubGoodMtg <- ggplot(EEFpred.PubGoodMtg, 
                              aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(1, 5) +
  ylab("Predicted Response on 1 to 5 scale") +
  xlab("Control Group Score") +
  ggtitle("Raise public goods issue at mtg") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Raise public goods issue w. official
EEFplot.PubGoodDirect <- ggplot(EEFpred.PubGoodDirect, 
                                 aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(1, 5) +
  ylab("Predicted Response on 1 to 5 scale") +
  xlab("Control Group Score") +
  ggtitle("Raise public goods issue w. official") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 12),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# Support pro-education candidate
EEFplot.Candidate <- ggplot(EEFpred.Candidate, 
                                 aes(x = EEFscore, y = est)) + 
  geom_line(size=1) +
  geom_errorbar(aes(ymin = as.numeric(as.character(cil)), 
                    ymax = as.numeric(as.character(ciu))), 
                width = 0, lwd = 1) +
  ylim(0, 1) +
  ylab("Predicted Probability") +
  xlab("Control Group Score") +
  ggtitle("Support pro-education candidate") +
  theme(panel.background = element_blank(),
        plot.title = element_text(size = 11),
        panel.border = element_rect(colour = "gray", fill=NA, size=.8),
        legend.position= "none")  

# combine plots
grid.arrange(EEFBplot.KFTotal_B, EEFBplot.CCK1_B_num, EEFBplot.EQ1_B_num, 
             EEFBplot.PP2_B_num, EEFBplot.CB1_B_num, EEFBplot.EQ5_B_bi, 
             EEFplot.Candidate, EEFplot.PubGoodMtg, EEFplot.PubGoodDirect, 
             nrow = 3)

@

While we cannot interpret any causal relations from such associations, these initial findings are at least consistent with the proposition that self-efficacy positively affects active citizen behaviors and development outcomes. Moreover, from a measurement perspective it is reassuring to find these patterns, especially with the child assessment scores which were measured independently from the parent surveys and thus, not subject to social desirability bias. 


\section{Experimental Results}
\subsection{Study 1: Effects on Parent and Student Outcomes in Bukoba Pilot}

<<VP_EEF_Buk, eval = TRUE, echo = FALSE, tidy=TRUE, warning=FALSE, message=FALSE, strip.white=TRUE, results='asis'>>=

# Fit the linear mixed effects models, regressing efficacy outcomes on interventions and control vars
# FULL SAMPLE, WEIGHTED

# ESES score
EEFfit <- lm_robust(EEFscore_E ~ VP + IW + EEFscore_B + 
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

EEFbase <- round(mean(studyBuk$EEFscore_B, na.rm = T), digits = 2) #mean and sd at baseline
EEFbasesd <- round(sd(studyBuk$EEFscore_B, na.rm = T), digits = 2)

# Internal ESES
EEFfitint <- lm_robust(EEFinternal_E ~ VP + IW + EEFinternal_B + 
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

EEFbaseint <- round(mean(studyBuk$EEFinternal_B, na.rm = T), digits = 2) #mean and sd at baseline
EEFbaseintsd <- round(sd(studyBuk$EEFinternal_B, na.rm = T), digits = 2)

# External ESES
EEFfitext <- lm_robust(EEFexternal_E ~ VP + IW + EEFexternal_B + 
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

EEFbaseext <- round(mean(studyBuk$EEFexternal_B, na.rm = T), digits = 2) #mean and sd at baseline
EEFbaseextsd <- round(sd(studyBuk$EEFexternal_B, na.rm = T), digits = 2)

# Individual ESES
EEFfitindiv <- lm_robust(EEFindiv_E ~ VP + IW + EEFindiv_B +  
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

EEFbaseindiv <- round(mean(studyBuk$EEFindiv_B, na.rm = T), digits = 2) #mean and sd at baseline
EEFbaseindivsd <- round(sd(studyBuk$EEFindiv_B, na.rm = T), digits = 2)

# Group ESES
EEFfitgroup <- lm_robust(EEFgroup_E ~ VP + IW + EEFgroup_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

EEFbasegroup <- round(mean(studyBuk$EEFgroup_B, na.rm = T), digits = 2) #mean and sd at baseline
EEFbasegroupsd <- round(sd(studyBuk$EEFgroup_B, na.rm = T), digits = 2)

# GSES
GEFfit <- lm_robust(GEFscore_E ~ VP + IW + GEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1, 
             clusters = SchoolID,
             data = studyBuk)

GEFbase <- round(mean(studyBuk$GEFscore_B, na.rm = T), digits = 2) #mean and sd at baseline
GEFbasesd <- round(sd(studyBuk$GEFscore_B, na.rm = T), digits = 2)

@

<<VP_Beh_Buk, eval = TRUE, echo = FALSE, tidy=TRUE, warning=FALSE, message=FALSE, strip.white=TRUE, results='asis'>>=

#Fit the linear mixed effects models, regressing efficacy outcomes on interventions and control vars
# FULL SAMPLE, WEIGHTED

# Behavioral Outcomes
                          
# Outcome: Believes child will complete secondary school
EQ5_E.fit <- lm_robust(EQ5_E_bi ~ VP + IW + 
                            EQ5_B_bi + EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

EQ5base <- round(mean(studyBuk$EQ5_B_num, na.rm = T), digits = 2) #mean and sd at baseline
EQ5basesd <- round(sd(studyBuk$EQ5_B_num, na.rm = T), digits = 2)

# Outcome: Asked about teacher absence last week?
EQ1_E.fit <- lm_robust(EQ1_E_num ~ VP + IW + 
                          EQ1_B_num + EEFscore_B + 
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

EQ1base <- round(mean(studyBuk$EQ1_B_num, na.rm = T), digits = 2) #mean and sd at baseline
EQ1basesd <- round(sd(studyBuk$EQ1_B_num, na.rm = T), digits = 2)

# Outcome: Looked at HW last week?
PP2_E.fit <- lm_robust(PP2_E_num ~ VP + IW + 
                          PP2_B_num + EEFscore_B + 
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

PP2base <- round(mean(studyBuk$PP2_B_num, na.rm = T), digits = 2) #mean and sd at baseline
PP2basesd <- round(sd(studyBuk$PP2_B_num, na.rm = T), digits = 2)

# Outcome: Child had breakfast today?
CB1_E.fit <- lm_robust(CB1_E_num ~ VP + IW + 
                          CB1_B_num + EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

CB1base <- round(mean(studyBuk$CB1_B_num, na.rm = T), digits = 2) #mean and sd at baseline
CB1basesd <- round(sd(studyBuk$CB1_B_num, na.rm = T), digits = 2)

# Outcome: Speak with teacher (endline only)
CP1_E.fit <- lm_robust(CP1_E_num ~ VP + IW + 
                         EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

CP1end <- round(mean(studyBuk$CP1_E_num, na.rm = T), digits = 2) #mean and sd at baseline
CP1endsd <- round(sd(studyBuk$CP1_E_num, na.rm = T), digits = 2)

# Outcome: Speak with parent (endline only)
FU1_E_bi.fit <- lm_robust(FU1_E_bi ~ VP + IW + 
                         EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

FU1end <- round(mean(studyBuk$FU1_E_bi, na.rm = T), digits = 2) #mean and sd at baseline
FU1endsd <- round(sd(studyBuk$FU1_E_bi, na.rm = T), digits = 2)

# Outcome: Test Scores
KFTotal_E.fit <- lm_robust(KFTotal_E ~ VP + IW + 
                         KFTotal_B + EEFscore_B +
             political_B + parent_B + muslim_B + 
             literate_B + HHhead_B + HH1_B +
             Female_B + econindexSUM_B + disabilities_B_bi +
             CH1_B + BC5_B_num + AgeBin,
             weights = cbpsweights,
             alpha = .1,
             clusters = SchoolID,
             data = studyBuk)

KFTotalbase <- round(mean(studyBuk$KFTotal_B, na.rm = T), digits = 2) #mean and sd at baseline
KFTotalbasesd <- round(sd(studyBuk$KFTotal_B, na.rm = T), digits = 2)


@

<<VP_Results_Buk, eval = TRUE, echo = FALSE, tidy=TRUE, fig.width = 10, fig.height = 5, fig.align='center', fig.pos = 'H', out.width= "1\\linewidth", warning=FALSE, message=FALSE, fig.cap="Study 1 (N=598 parents; 24 school clusters): Average treatment effects of VP (blue) and IW (red) compared to control on endline efficacy and active citizenship outcomes (with 95 $\\%$ and 90$\\%$ CIs). OLS models controlling for baseline measures and demographic covariates, with school cluster robust standard errors. VP--IW difference (black).">>=

## Combine effects for Efficacy
VP_EEF_Buk.combine <- as.data.frame(rbind(mytidy(EEFfit),
                                  mytidy(EEFfitint),
                                  mytidy(EEFfitext),
                                  mytidy(EEFfitindiv),
                                  mytidy(EEFfitgroup)))

VP_EEF_Buk.combine <- VP_EEF_Buk.combine[VP_EEF_Buk.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_EEF_Buk.combine$outcome2 <- c(rep("Efficacy Score", 3),
                                 rep("Internal Efficacy", 3),
                                 rep("External Efficacy", 3),
                                 rep("Individual Efficacy", 3),
                                 rep("Group Efficacy", 3))

VP_EEF_Buk.combine$order <- seq(from = nrow(VP_EEF_Buk.combine), to = 1, by = -1)

## Generate plots for Efficacy
VP_EEF_Buk.plot <- ggplot(VP_EEF_Buk.combine, 
                          aes(x = reorder(outcome2, order), 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.35, .4) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    annotate("text", x=5.18, y=-.29, label= "VP-control", color = "#0072B2", size=2.8) + 
    annotate("text", x=5, y=-.29, label= "IW-control", color = "#ba0000", size=2.8) + 
    annotate("text", x=4.8, y=-.30, label= "VP-IW", color = "black", size=2.8) + 
    ggtitle("Efficacy Scores") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

## Combine effects for School Behaviors 
VP_Beh1_Buk.combine <- as.data.frame(rbind(mytidy(EQ1_E.fit),
                                  mytidy(CP1_E.fit),
                                  mytidy(FU1_E_bi.fit)))

VP_Beh1_Buk.combine <- VP_Beh1_Buk.combine[VP_Beh1_Buk.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_Beh1_Buk.combine$outcome2 <- c(rep("Asked about teacher absence", 3),
                                 rep("Spoke with teacher last week", 3),
                                 rep("Wrote a comment for head teacher", 3))

VP_Beh1_Buk.combine$order <- seq(from = nrow(VP_Beh1_Buk.combine), to = 1, by = -1)

VP_Beh1_Buk.plot <- ggplot(VP_Beh1_Buk.combine, 
                           aes(x = reorder(outcome2, order), 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.35, .4) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    ggtitle("Teacher-Oriented Actions (0 no, 1 yes)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

VP_Beh2_Buk.combine <- as.data.frame(rbind(mytidy(PP2_E.fit),
                                  mytidy(CB1_E.fit),
                                  mytidy(EQ5_E.fit)))

VP_Beh2_Buk.combine <- VP_Beh2_Buk.combine[VP_Beh2_Buk.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_Beh2_Buk.combine$outcome2 <- c(rep("Looked at HW last week", 3),
                                 rep("Gave breakfast today", 3),
                                 rep("Certain graduate secondary", 3))

VP_Beh2_Buk.combine$order <- seq(from = nrow(VP_Beh2_Buk.combine), to = 1, by = -1)

VP_Beh2_Buk.plot <- ggplot(VP_Beh2_Buk.combine, 
                           aes(x = reorder(outcome2, order), 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = NULL) +
    ylim(-.35, .4) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    ggtitle("Child-Oriented Actions (0 no, 1 yes)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))


## Effects for Test Scores
VP_KFTotal_Buk.combine <- as.data.frame(mytidy(KFTotal_E.fit))
                              
VP_KFTotal_Buk.combine <- VP_KFTotal_Buk.combine[VP_KFTotal_Buk.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_KFTotal_Buk.combine$outcome2 <- c(rep("Test\nScore", 3))

VP_KFTotal_Buk.plot <- ggplot(VP_KFTotal_Buk.combine, 
                           aes(x = outcome2, 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-7, 5) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    ggtitle("Child's Test Score (/72 pts)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

## Show plots
stacked <- VP_Beh2_Buk.plot/VP_KFTotal_Buk.plot + plot_layout(heights = c(2.5, 1))

VP_EEF_Buk.plot + VP_Beh1_Buk.plot + stacked + plot_layout(widths = c(1, 1, 1))


@

Turning to our experimental findings, Figure \ref{fig:VP_Results_Buk} summarizes the results from Study 1 in Bukoba.\footnote{All regression tables are provided in SI Section \ref{sec:SIregtables}.} Since we pre-registered the positive direction of predicted treatment effects, we report 90\% confidence intervals based on a two-sided test, the lower-bound of which is equivalent to that of a 95\% confidence interval for a one-sided test. With respect to efficacy, while the estimates for both VP and IW are positive compared to control, all estimated effects are modest, and none are statistically different from zero at alpha = .05. For parental behaviors, VP increased the probability for asking about teacher absence by \Sexpr{VP_Beh1_Buk.combine[VP_Beh1_Buk.combine$term == "VP" & VP_Beh1_Buk.combine$outcome == "EQ1_E_num",]$estimate*100} percentage points (90\% CI = [\Sexpr{VP_Beh1_Buk.combine[VP_Beh1_Buk.combine$term == "VP" & VP_Beh1_Buk.combine$outcome == "EQ1_E_num",]$conf.low*100}, \Sexpr{VP_Beh1_Buk.combine[VP_Beh1_Buk.combine$term == "VP" & VP_Beh1_Buk.combine$outcome == "EQ1_E_num",]$conf.high*100}]); and looking at homework by \Sexpr{VP_Beh2_Buk.combine[VP_Beh2_Buk.combine$term == "VP" & VP_Beh2_Buk.combine$outcome == "PP2_E_num",]$estimate*100} percentage points (90\% CI = [\Sexpr{VP_Beh2_Buk.combine[VP_Beh2_Buk.combine$term == "VP" & VP_Beh2_Buk.combine$outcome == "PP2_E_num",]$conf.low*100}, \Sexpr{VP_Beh2_Buk.combine[VP_Beh2_Buk.combine$term == "VP" & VP_Beh2_Buk.combine$outcome == "PP2_E_num",]$conf.high*100}]). We find similar estimates for IW such that the difference between VP and IW (shown in black) is essentially zero. In our pre-analysis plan, even with more optimistic predictions about self-efficacy, we indicated that we did not expect to see treatment effects with respect to student test scores, and indeed, that is what we find.

Study 1 was expressly designed as a ``pilot,'' and at its conclusion, we reflected on potential shortcomings in design and implementation that we could address in follow-up research. First, we surmised that the active involvement of live facilitators in the IW meetings may have resulted in IW respondents receiving some ``dose'' of VP, which would have impeded our ability to estimate differences between these treatment arms, but would not have explained the generally weak effects overall. Second, because of the similarity of baseline and endline questionnaires, endline responses may have been a function of subjects' attempts to recall how they answered at baseline. Third, because of the relatively small sample in the pilot, there was some important imbalance in the pre-treatment distribution of covariates, including higher baseline efficacy in the VP and control groups relative to IW.\footnote{As a robustness check, we use the OLS interacted adjustment method proposed by \citet{Lin:2013} to address possible concerns about baseline imbalance of efficacy scores in Study 1 and heterogeneous treatment effects that may be related to covariates. SI Section \ref{sec:SIlinadjust} shows that under this alternative specification, our findings do not substantively change.} All of these factors may have weighed against detecting the full extent of expected treatment effects, and we sought to make adjustments in our second study in Kilosa.

Despite the estimated null findings, we note that we found important differences in facilitator notes concerning the content of what parents discussed in IW as compared with VP meetings. During the IW workshops, parents were curious about what the sponsoring NGO, Twaweza, would do to help solve school problems after the meetings ended.\footnote{We had not discussed this possible outcome with our facilitators, nor was it part of the meeting scripts.} By contrast, in the VP meetings, parents expressed interest in volunteering and initiating peer networks with other parents who had not been involved in the study, ideas and actions that reflected greater inclinations towards active citizenship. SI Section \ref{sec:SIbukobaqual} discusses this qualitative evidence in more detail. 


\subsection{Study 1 Follow-up with Teachers on School-Level Outcomes} 

<<Qual1results, eval = TRUE, echo = FALSE, tidy=TRUE, fig.width = 7, fig.height = 7, out.width= ".9\\linewidth", fig.align='center', warning=FALSE, message=FALSE, strip.white=TRUE, results='hide'>>=

### Qualitative data analysis: proportion of positive versus neutral comments

## Compare head teachers, classroom teachers, and parents together
qualtestall <- prop.table(table(qualdat[qualdat$Who %in% c("HT", "CT", "par"),]$treat,
                                qualdat[qualdat$Who %in% c("HT", "CT", "par"),]$evidence), 1)

qualtestall.ttest.VPIW <- t.test(qualdat[qualdat$Who %in% c("HT", "CT", "par") & qualdat$treat == "VP",]$evidence,
                                 qualdat[qualdat$Who %in% c("HT", "CT", "par") & qualdat$treat == "IW",]$evidence)
# p-value = 0.09889

qualtestall.ttest.VPSO <- t.test(qualdat[qualdat$Who %in% c("HT", "CT", "par") & qualdat$treat == "VP",]$evidence,
                                 qualdat[qualdat$Who %in% c("HT", "CT", "par") & qualdat$treat == "SO",]$evidence)
# p-value = 0.008127

## Compare just parents 
qualtestpar <- prop.table(table(qualdat[qualdat$Who %in% c("par"),]$treat,
                                qualdat[qualdat$Who %in% c("par"),]$evidence), 1)

qualtestpar.ttest.VPIW <- t.test(qualdat[qualdat$Who %in% c("par") & qualdat$treat == "VP",]$evidence,
                                 qualdat[qualdat$Who %in% c("par") & qualdat$treat == "IW",]$evidence)
# p-value = 0.4632

qualtestpar.ttest.VPSO <- t.test(qualdat[qualdat$Who %in% c("par") & qualdat$treat == "VP",]$evidence,
                                 qualdat[qualdat$Who %in% c("par") & qualdat$treat == "SO",]$evidence)
# p-value = 0.8931

## Compare teachers
qualtesttch <- prop.table(table(qualdat[qualdat$Who %in% c("HT", "CT"),]$treat,
                                qualdat[qualdat$Who %in% c("HT", "CT"),]$evidence), 1)

qualtesttch.ttest.VPIW <- t.test(qualdat[qualdat$Who %in% c("HT", "CT") & qualdat$treat == "VP",]$evidence,
                                 qualdat[qualdat$Who %in% c("HT", "CT") & qualdat$treat == "IW",]$evidence)
# p-value = 0.01561

qualtesttch.ttest.VPSO <- t.test(qualdat[qualdat$Who %in% c("HT", "CT") & qualdat$treat == "VP",]$evidence,
                                 qualdat[qualdat$Who %in% c("HT", "CT") & qualdat$treat == "SO",]$evidence)
# p-value = 8.467e-05

@

Beyond beliefs and attitudes as self-reported on surveys, we sought to learn whether the treatment had any impact on Study 1 schools in the medium- and long-term. In the absence of detailed administrative records concerning parental engagement and grades of the children of our study parents, we took advantage of the Study 1 research design by returning to schools to interview multiple teachers and head teachers with respect to a number of dimensions of parental involvement and student outcomes six months and then two years post-treatment. While such data are clearly inferior to directly-observed parent behaviors and school interactions, such interviews offered a relatively low-cost alternative, and provides insight into the perspectives of key actors in this sector. 

First, for the six-month post-study assessment, we directed research assistants to conduct interviews with 24 teachers from a randomly selected number of Study 1 schools using a structured guide, asking them about their impressions of parents' behaviors at their schools. We then transcribed these teacher interviews and independently coded them blind to treatment status by whether there was positive change (i.e., more pro-education actions by parents observed in the past few months), no change, or negative change in parents' behaviors relative to the desired outcome of more active engagement. We found that \Sexpr{qualtesttch[3,2]*100}\% of teacher comments indicated positive change in the VP schools, but for IW and control schools, the proportion of positive teacher comments were much less at \Sexpr{qualtesttch[1,2]*100}\% and \Sexpr{qualtesttch[2,2]*100}\%, respectively.\footnote{The p-values from simple two-sided T-tests comparing the proportion of positive VP and IW teacher comments and the proportion of positive VP and control teacher comments are \Sexpr{qualtesttch.ttest.VPIW$p.value} and \Sexpr{qualtesttch.ttest.VPSO$p.value}, respectively.} We provide examples of those comments and our coding scheme in SI Section \ref{sec:SIbukobaqual}.

<<Qual2results, eval = TRUE, echo = FALSE, tidy=TRUE, fig.width = 10, fig.height = 4.5, fig.align='center', fig.pos = 't', out.width= "1\\linewidth", warning=FALSE, message=FALSE, fig.cap="Downstream Follow-Up in Study 1 Schools, two years later (N=117 teachers; 24 school clusters): Average treatment effects of VP (blue) and IW (red) compared to control on parental active citizenship outcomes 2 years later as reported on structured assessments with teachers (with 95$\\%$ and 90$\\%$ CIs). OLS models with respondent type FE and school cluster robust standard errors. VP--IW difference (black).">>=

## Run models

## 3 point outcomes on levels of improvement

# change_edu_outcomes
# -1: doing worse; 0: stayed the same; 1: improved
change_edu_outcomes <- lm_robust(change_edu_outcomes ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# change_edu_resources
# -1: doing worse; 0: stayed the same; 1: improved
change_edu_resources <- lm_robust(change_edu_resources ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# different_par_stan5
# -1: standard 5 parents less involved; 0: no difference: 1: standard 5 parents more involved
different_par_stan5 <- lm_robust(different_par_stan5 ~ VP + IW,
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# past_study_students
# -1: doing worse; 0: doing same; 1: doing better
past_study_students <- lm_robust(past_study_students ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# response_localgov
# -1: doing worse; 0: stayed the same; 1: improved; NA
response_localgov <- lm_robust(response_localgov ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

## 3 point outcomes on one or many

# actions_parent_group
# 0: teacher does not describe any actions; 1: teacher describes just one isolated action; 2: pattern of action / multiple actions
actions_parent_group <- lm_robust(actions_parent_group ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# parent_attend_speakup_3
# 0: few parents speak; 1: some parents speak; 2: many parents speak
parent_attend_speakup_3 <- lm_robust(parent_attend_speakup_3 ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# parent_interactions_2yrs
# 0: answer has no notable actions; 1: just one isolated action; 2: pattern of action / multiple actions
parent_interactions_2yrs <- lm_robust(parent_interactions_2yrs ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# parentmeeting_last_1
# 0: no meeting; 1: meeting but parents did not speak; 2: parents came and spoke up
parentmeeting_last_1 <- lm_robust(parentmeeting_last_1 ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

## Binary outcomes

# contributions_parents_financial
# 0: no or very low contributions; 1: yes some financial contributions
contributions_parents_financial <- lm_robust(contributions_parents_financial ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# contributions_parents_other
# 0: no or very low contributions; 1: not financial but any other form of contribution mentioned
contributions_parents_other <- lm_robust(contributions_parents_other ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# contributions_parents_any
# 0: no or very low contributions; 1: any contributions mentioned, financial or otherwise
contributions_parents_any <- lm_robust(contributions_parents_any ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# help_parents
# 0: parents not helpful; 1: parents helpful in at least some ways
help_parents <- lm_robust(help_parents ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# parent_type
# 0: mostly elder men; 1: others speak up about the same
parent_type <- lm_robust(parent_type ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)

# parent_attend_speakup_4
# 0: mostly elder men speak; 1: others speak up about the same
parent_attend_speakup_4 <- lm_robust(parent_attend_speakup_4 ~ VP + IW, 
                       data = qual2[qual2$respondenttype2 != "Research Assistant",],
                       fixed_effects = ~respondenttype2,
                       alpha = .1, 
                       clusters = schoolname)


## Combine effects
qual2effects_improvement <- as.data.frame(rbind(mytidy(change_edu_outcomes),
                                  #tidy(change_edu_resources),
                                  #tidy(different_par_stan5),
                                  mytidy(past_study_students),
                                  mytidy(response_localgov)))

qual2effects_improvement$outcome2 <- c(rep("Educational outcomes", 3),
                                 rep("Study students", 3),
                                 rep("Local government responsiveness", 3))

qual2effects_improvement$order <- seq(from = nrow(qual2effects_improvement), to = 1, by = -1)

qual2effects_multiple <- as.data.frame(rbind(mytidy(parent_interactions_2yrs),
                                  mytidy(parentmeeting_last_1),
                                  mytidy(actions_parent_group)))

qual2effects_multiple$outcome2 <- c(rep("Teacher parent interactions", 3),
                                    rep("Parents participate in mtgs", 3),
                                    rep("Parents taking notable actions", 3))

qual2effects_multiple$order <- seq(from = nrow(qual2effects_multiple), to = 1, by = -1)


qual2effects_binary <- as.data.frame(rbind(
                                  #tidy(contributions_parents_financial),
                                  #tidy(contributions_parents_other),
                                  mytidy(contributions_parents_any),
                                  mytidy(help_parents),
                                  #tidy(parent_type),
                                  mytidy(parent_attend_speakup_4)))

qual2effects_binary$outcome2 <- c(rep("Parents make contributions", 3),
                                 rep("Parents help with schoolwork", 3),
                                 rep("Diverse parents speaking", 3))

qual2effects_binary$order <- seq(from = nrow(qual2effects_binary), to = 1, by = -1)

## Generate plots
qual2plot_multiple <- ggplot(qual2effects_multiple, 
                             aes(x = reorder(outcome2, order),
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.7, 1) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    annotate("text", x=3.2, y=-.50, label= "VP-control", color = "#0072B2", size=3) + 
    annotate("text", x=3.1, y=-.50, label= "IW-control", color = "#ba0000", size=3) + 
    annotate("text", x=2.85, y=-.55, label= "VP-IW", color = "black", size=3) + 
    ggtitle("Frequency in past 2 years\n(0 none, 1 some, 2 multiple)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

qual2plot_improvement <- ggplot(qual2effects_improvement, 
                                aes(x = reorder(outcome2, order),
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.7, 1) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    ggtitle("Change in past 2 years\n(-1 worse, 0 same, 1 better)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

qual2plot_binary <- ggplot(qual2effects_binary, 
                            aes(x = reorder(outcome2, order),
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.7, 1) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    ggtitle("Binary outcomes\n(0 no, 1 yes)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

## Show plots
grid.arrange(qual2plot_multiple, 
             qual2plot_binary,
             qual2plot_improvement,
             ncol = 3)

@


Approximately two years later, we again completed structured assessments with 117 teachers and head teachers in all 24 schools and independently coded their comments blind to treatment status. Compared to control schools, Figure \ref{fig:Qual2results} shows that teacher assessments of parent behaviors and of school-related outcomes are consistently more positive for the VP schools relative to IW schools. Several of these differences (shown in black) are statistically and substantively significant. First, the frequency of teacher-parent interactions in VP compared to control schools had improved by 
\Sexpr{qual2effects_multiple[qual2effects_multiple$term == "VP" & qual2effects_multiple$outcome == "parent_interactions_2yrs",]$estimate} (95\% CI = [\Sexpr{qual2effects_multiple[qual2effects_multiple$term == "VP" & qual2effects_multiple$outcome == "parent_interactions_2yrs",11]}, \Sexpr{qual2effects_multiple[qual2effects_multiple$term == "VP" & qual2effects_multiple$outcome == "parent_interactions_2yrs",12]}]) on a three-point scale from never to multiple times. Compared to IW schools, this difference is 
\Sexpr{qual2effects_multiple[qual2effects_multiple$term == "diff_vpiw" & qual2effects_multiple$outcome == "parent_interactions_2yrs",]$estimate} (95\% CI = [\Sexpr{qual2effects_multiple[qual2effects_multiple$term == "diff_vpiw" & qual2effects_multiple$outcome == "parent_interactions_2yrs",11]}, \Sexpr{qual2effects_multiple[qual2effects_multiple$term == "diff_vpiw" & qual2effects_multiple$outcome == "parent_interactions_2yrs",12]}]). 

With respect to private behaviors within the household, the likelihood of parents being helpful with children's schoolwork was greater in VP communities by a substantially large 
\Sexpr{qual2effects_binary[qual2effects_binary$term == "VP" & qual2effects_binary$outcome == "help_parents",]$estimate * 100} percentage points (95\% CI = [\Sexpr{qual2effects_binary[qual2effects_binary$term == "VP" & qual2effects_binary$outcome == "help_parents",11] * 100}, \Sexpr{qual2effects_binary[qual2effects_binary$term == "VP" & qual2effects_binary$outcome == "help_parents",12] * 100}]).\footnote{While parent-level data might have been preferable, we could not obtain these owing to cost and logistical constraints. However, because parent-level self-reporting of behaviors is subject to social desirability bias, we believe that the teacher-reported assessments do a better job of tracking overall patterns of parent behavior in the respective study groups.} Compared to IW schools, this difference is 
\Sexpr{qual2effects_binary[qual2effects_binary$term == "diff_vpiw" & qual2effects_binary$outcome == "help_parents",]$estimate * 100} percentage points (95\% CI = [\Sexpr{qual2effects_binary[qual2effects_binary$term == "diff_vpiw" & qual2effects_binary$outcome == "help_parents",11] * 100}, \Sexpr{qual2effects_binary[qual2effects_binary$term == "diff_vpiw" & qual2effects_binary$outcome == "help_parents",12] * 100}]). Additionally, parents were more likely to make contributions to VP schools compared to IW schools by 
\Sexpr{qual2effects_binary[qual2effects_binary$term == "diff_vpiw" & qual2effects_binary$outcome == "contributions_parents_any",]$estimate * 100} percentage points (95\% CI = [\Sexpr{qual2effects_binary[qual2effects_binary$term == "diff_vpiw" & qual2effects_binary$outcome == "contributions_parents_any",11] * 100}, \Sexpr{qual2effects_binary[qual2effects_binary$term == "diff_vpiw" & qual2effects_binary$outcome == "contributions_parents_any",12] * 100}]).
Finally, to assess educational outcomes, we asked ``Have education outcomes like student and teacher attendance, marks, and pass rates improved/stayed the same/worsened in the past 2 years? Please explain.'' We find that assessments of educational outcomes for VP schools were better than control schools by \Sexpr{qual2effects_improvement[qual2effects_improvement$term == "VP" & qual2effects_improvement$outcome == "change_edu_outcomes",]$estimate} (95\% CI = [\Sexpr{qual2effects_improvement[qual2effects_improvement$term == "VP" & qual2effects_improvement$outcome == "change_edu_outcomes",11]}, \Sexpr{qual2effects_improvement[qual2effects_improvement$term == "VP" & qual2effects_improvement$outcome == "change_edu_outcomes",12]}]) on a three-point scale from worse to better. Compared to IW schools, this difference is 
\Sexpr{qual2effects_improvement[qual2effects_improvement$term == "diff_vpiw" & qual2effects_improvement$outcome == "change_edu_outcomes",]$estimate} (95\% CI = [\Sexpr{qual2effects_improvement[qual2effects_improvement$term == "diff_vpiw" & qual2effects_improvement$outcome == "change_edu_outcomes",11]}, \Sexpr{qual2effects_improvement[qual2effects_improvement$term == "diff_vpiw" & qual2effects_improvement$outcome == "change_edu_outcomes",12]}]).\footnote{In Figure \ref{fig:Qual2results}, outcomes of the right and left panels are coded on a three-point scale. When recoded as binary outcomes as specified in our pre-analysis plan, results do not substantively change (see SI Section \ref{sec:SIQual2results_binary}).}

These findings, while at odds with null effects detected on our parent endline surveys, provide some suggestive evidence that Validated Participation may have longer-term, positive consequences for the ``co-production'' of better educational outcomes'' \citep{Ostrom:1996,Bebbington:1999,Woolcock:2000}. But how might we reconcile these differential findings? It may be that there were small differences at the endline, yet our data were simply too sparse to precisely estimate them. Consistent with our theory shown in Figure \ref{fig:theory}, these small differences in efficacy and engagement may have magnified over time through the ``virtuous cycle,'' generating more mastery experiences, greater efficacy beliefs, engagement, and so on. Particularly if VP parents mobilized other non-study parents in their schools, as qualitatively reported, these changes in behavior would multiply and be visible at the school level.
As we discuss below, the consistency of the findings in the teacher study with our theoretical predictions leads us ultimately to conclude that the findings from the endline parent study are not fully dispositive with respect to our theory.


\subsection{Study 2: Effects on Parent Outcomes in Kilosa}

<<VP_EEF_Kil, eval = TRUE, echo = FALSE, tidy=TRUE, warning=FALSE, message=FALSE, strip.white=TRUE, results='asis'>>=

EEFfit.kil <- lm_robust(EEFscore ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

EEFso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$EEFscore, na.rm = T), digits = 2) #mean and sd for SO
EEFsosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$EEFscore, na.rm = T), digits = 2)

EEFfitint.kil <- lm_robust(EEFinternal ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

EEFsoint <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$EEFinternal, na.rm = T), digits = 2) #mean and sd for SO
EEFsointsd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$EEFinternal, na.rm = T), digits = 2)

EEFfitext.kil <- lm_robust(EEFexternal ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

EEFsoext <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$EEFexternal, na.rm = T), digits = 2) #mean and sd for SO
EEFsoextsd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$EEFexternal, na.rm = T), digits = 2)

EEFfitindiv.kil <- lm_robust(EEFindiv ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

EEFsoindiv <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$EEFindiv, na.rm = T), digits = 2) #mean and sd for SO
EEFsoindivsd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$EEFindiv, na.rm = T), digits = 2)

EEFfitgroup.kil <- lm_robust(EEFgroup ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

EEFsogroup <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$EEFgroup, na.rm = T), digits = 2) #mean and sd for SO
EEFsogroupsd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$EEFgroup, na.rm = T), digits = 2)

@

<<VP_Beh_Kil, eval = TRUE, echo = FALSE, tidy=TRUE, fig.width = 7, fig.height = 7, out.width= ".9\\linewidth", fig.pos = 't!', fig.align='center', warning=FALSE, message=FALSE, strip.white=TRUE, results='asis'>>=

# Outcome: Respected by Parents
RespectedbyParents.fit <- lm_robust(RespectedbyParents ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

RespectedbyParentsso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$RespectedbyParents, 
                                     na.rm = T), digits = 2) #mean and sd for SO
RespectedbyParentssosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$RespectedbyParents, 
                                     na.rm = T), digits = 2)

# Outcome: Respected by Authorities
RespectedbyAuth.fit <- lm_robust(RespectedbyAuth ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

RespectedbyAuthso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$RespectedbyAuth, 
                                     na.rm = T), digits = 2) #mean and sd for SO
RespectedbyAuthsosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$RespectedbyAuth, 
                                     na.rm = T), digits = 2)

# Outcome: SC Ideas Respected
RespectedIdeas.fit <- lm_robust(RespectedIdeas ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

RespectedIdeasso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$RespectedIdeas, 
                                     na.rm = T), digits = 2) #mean and sd for SO
RespectedIdeassosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$RespectedIdeas, 
                                     na.rm = T), digits = 2)

# Outcome: Public Goods Mtg
PubGoodMtg.fit <- lm_robust(PubGoodMtg ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

PubGoodMtgso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$PubGoodMtg, 
                                     na.rm = T), digits = 2) #mean and sd for SO
PubGoodMtgsosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$PubGoodMtg, 
                                     na.rm = T), digits = 2)

# Outcome: Public Goods Direct
PubGoodDirect.fit <- lm_robust(PubGoodDirect ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

PubGoodDirectso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$PubGoodDirect, 
                                     na.rm = T), digits = 2) #mean and sd for SO
PubGoodDirectsosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$PubGoodDirect, 
                                     na.rm = T), digits = 2)

# Outcome: Support Pro-Education Candidate
Candidate.fit <- lm_robust(Candidate ~ VP + IW + morning +
               Female + HHhead + Kiswahili + Parent +
               Literate + Muslim + Political + Wakaguru +
               hh_size + AgeBin + Children + 
               Standard + econindexSUM +
               Education_num + 
               as.factor(schoolID),
               clusters = schoolID,
               alpha = .1, 
               data = studyKil)

Candidateso <- round(mean(studyKil[studyKil$ClassID == "Survey Only",]$Candidate, 
                                     na.rm = T), digits = 2) #mean and sd for SO
Candidatesosd <- round(sd(studyKil[studyKil$ClassID == "Survey Only",]$Candidate, 
                                     na.rm = T), digits = 2)

@

<<VP_Results_Kil, eval = TRUE, echo = FALSE, tidy=TRUE, fig.width = 10, fig.height = 5, fig.align='center', fig.pos = 't', out.width= "1\\linewidth", warning=FALSE, message=FALSE, fig.cap="Study 2 (N=1633 parents; individual-level treatment assignment across 16 schools): Average treatment effects of VP (blue) and IW (red) compared to control on same-day, post-meeting efficacy, pro-education political behaviors, and feeling respected (with 95$\\%$ and 90$\\%$ CIs). OLS models controlling for demographic covariates, with school fixed effects and cluster robust standard errors. VP--IW difference (black).">>=

## Generate plots for Efficacy
VP_EEF_Kil.combine <- as.data.frame(rbind(mytidy(EEFfit.kil),
                                  mytidy(EEFfitint.kil),
                                  mytidy(EEFfitext.kil),
                                  mytidy(EEFfitindiv.kil),
                                  mytidy(EEFfitgroup.kil)))

VP_EEF_Kil.combine <- VP_EEF_Kil.combine[VP_EEF_Kil.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_EEF_Kil.combine$outcome2 <- c(rep("Efficacy Score", 3),
                                 rep("Internal Efficacy", 3),
                                 rep("External Efficacy", 3),
                                 rep("Individual Efficacy", 3),
                                 rep("Group Efficacy", 3))

VP_EEF_Kil.combine$order <- seq(from = nrow(VP_EEF_Kil.combine), to = 1, by = -1)

VP_EEF_Kil.plot <- ggplot(VP_EEF_Kil.combine, 
                          aes(x = reorder(outcome2, order), 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.1, .2) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    annotate("text", x=5.18, y=-.068, label= "VP-control", color = "#0072B2", size=2.8) + 
    annotate("text", x=5, y=-.068, label= "IW-control", color = "#ba0000", size=2.8) + 
    annotate("text", x=4.8, y=-.065, label= "VP-IW", color = "black", size=2.8) + 
    ggtitle("Effects on Same Day\nEfficacy Scores") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

## Combine effects for School Behaviors 
VP_Beh1_Kil.combine <- as.data.frame(rbind(mytidy(Candidate.fit),
                                  mytidy(PubGoodMtg.fit),
                                  mytidy(PubGoodDirect.fit)))

VP_Beh1_Kil.combine <- VP_Beh1_Kil.combine[VP_Beh1_Kil.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_Beh1_Kil.combine$outcome2 <- c(rep("Support pro-edu candidate (0 no, 1 yes)", 3),
                                 rep("Raise public goods issue at mtg", 3),
                                 rep("Raise public goods issue w. official", 3))

VP_Beh1_Kil.combine$order <- seq(from = nrow(VP_Beh1_Kil.combine), to = 1, by = -1)

VP_Beh1_Kil.plot <- ggplot(VP_Beh1_Kil.combine, 
                           aes(x = reorder(outcome2, order), 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
    geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.13, .25) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 13)) +
    ggtitle("Effects on Pro-Education\nPolitical Behaviors (1 - 5 scale)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

VP_Beh2_Kil.combine <- as.data.frame(rbind(mytidy(RespectedbyParents.fit),
                                  mytidy(RespectedbyAuth.fit),
                                  mytidy(RespectedIdeas.fit)))

VP_Beh2_Kil.combine <- VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term %in% c("VP", "IW", "diff_vpiw"),]

VP_Beh2_Kil.combine$outcome2 <- c(rep("Respected by other parents", 3),
                                 rep("Respected by authorities", 3),
                                 rep("Ideas respected by school committee", 3))

VP_Beh2_Kil.combine$order <- seq(from = nrow(VP_Beh2_Kil.combine), to = 1, by = -1)

VP_Beh2_Kil.plot <- ggplot(VP_Beh2_Kil.combine, 
                           aes(x = reorder(outcome2, order), 
                               y = estimate, 
                               group = term,
                               color = term)) +
    geom_hline(yintercept = 0, color = "gray80", lty = "dashed") +
    geom_point(position=position_dodge(width = .5), 
               aes(shape=term), size = 2.5) +
   geom_errorbar(aes(ymin=conf.low, ymax=conf.high), 
                  position=position_dodge(width = .5),
                   width=0, lwd = 1.5) +
    geom_errorbar(aes(ymin=`2.5 %`, ymax=`97.5 %`), 
                  position=position_dodge(width = .5),
                   width=0, lwd = .5) +
    scale_colour_manual(values=c("black", "#ba0000", "#0072B2")) +
    coord_flip() +
    labs(x = NULL, 
         y = "Average Treatment Effects") +
    ylim(-.17, .29) +
    scale_x_discrete(labels = function(x) str_wrap(x, width = 10)) +
    ggtitle("Effects on Feeling Respected\n(1 - 5 scale)") +
    theme(panel.background = element_blank(),
          legend.title = element_blank(), 
          plot.title = element_text(size = 10),
          panel.border = element_rect(colour = "gray", fill=NA, size=.8),
          legend.position = "none",
          axis.text.x  = element_text(angle=0, vjust=.5, hjust = .5, size=11))

## Show plots
grid.arrange(VP_EEF_Kil.plot, 
             VP_Beh1_Kil.plot, 
             VP_Beh2_Kil.plot, ncol = 3)

@

We implemented Study 2 -- a same-day individual-level study of parents -- with attention to the aforementioned concerns raised in Study 1. Figure \ref{fig:VP_Results_Kil} shows, as predicted, that VP leads to a modest increase in parents' educational self-efficacy. Interestingly, the effects of VP were larger in the case of the overall efficacy score by \Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFscore",]$estimate} (90\% CI = [\Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFscore",]$conf.low}, \Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFscore",]$conf.high}]), external efficacy by \Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFexternal",]$estimate} (90\% CI = [\Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFexternal",]$conf.low}, \Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFexternal",]$conf.high}]), and group efficacy by \Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFgroup",]$estimate} (90\% CI = [\Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFgroup",]$conf.low}, \Sexpr{VP_EEF_Kil.combine[VP_EEF_Kil.combine$term == "VP" & VP_EEF_Kil.combine$outcome == "EEFgroup",]$conf.high}]); smaller for individual and internal efficacy, and in the latter two cases, the effects of the VP treatment fall below statistical significance.\footnote{Again, we pre-registered the positive direction of predicted treatment effects, thus we report 90\% confidence intervals.} 

In the case of group efficacy, the single intervention caused a \Sexpr{EEFfitgroup.kil$coefficients[2]/EEFsogroupsd} standard deviation increase in self-reported beliefs. As a highly social intervention, the effects appear most promising for socially- and externally-oriented outcomes. While the individuals who received the IW also report higher levels of self-efficacy than the control, the estimated difference is smaller, and not statistically different from zero. In turn, the differences between IW and VP groups are close to zero and not statistically significant. 

We also examined the effects of treatment on questions about likely pro-education political behaviors and feelings of respect from others. In all cases, the effects from VP were at least as large and in the predicted direction as compared with Information Workshop. VP parents reported statistically significant positive effects for feeling respected by other parents (\Sexpr{VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term == "VP" & VP_Beh2_Kil.combine$outcome == "RespectedbyParents",]$estimate}, 95\% CI = [\Sexpr{VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term == "VP" & VP_Beh2_Kil.combine$outcome == "RespectedbyParents",11]}, \Sexpr{VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term == "VP" & VP_Beh2_Kil.combine$outcome == "RespectedbyParents",12]}]) and by authorities (\Sexpr{VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term == "VP" & VP_Beh2_Kil.combine$outcome == "RespectedbyAuth",]$estimate}, 95\% CI = [\Sexpr{VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term == "VP" & VP_Beh2_Kil.combine$outcome == "RespectedbyAuth",11]}, \Sexpr{VP_Beh2_Kil.combine[VP_Beh2_Kil.combine$term == "VP" & VP_Beh2_Kil.combine$outcome == "RespectedbyAuth",12]}]).\footnote{With respect to multiple hypothesis testing, we use the Benjamini-Hochberg (BH) adjusted p-values for each test of our findings. Given the more stringent adjusted p-values, all the statistically significant effects of VP retain their significance except the three efficacy outcomes for Study 2. See Section \ref{sec:SImulthypothesis} in the SI.} However, in no cases were the effects of VP substantively or statistically different from IW.


\section{Discussion}

This project contributes to longstanding questions about what might encourage or discourage citizen engagement in civic affairs, particularly in developing country settings. Self-efficacy beliefs are a logical gateway for active citizenship, and in our observational (non-experimental) analyses, we find that such beliefs strongly predict such behaviors, while recognizing that in practice, there is likely a feedback between the two. Such correlations are suggestive that there may indeed by an ``efficacy trap,'' but the bigger question around which our studies were framed concerned whether people could be nudged into feeling a heightened sense of self-efficacy when treated with external validation of their ideas and actions, especially as compared with citizens who were treated only with relevant information.

In most of our analyses, we find that our experimental intervention did not substantially increase parent-reported feelings of self-efficacy or active citizenship behaviors, especially relative to those parents who received a more conventional informational intervention. And as such, our study contributes more evidence to a large body of scholarship investigating the role that information and modified informational interventions may play in democratic accountability towards human development. Nonetheless, we do note some important exceptions detected in our qualitative research, including from enumerator notes and semi-structured interviews with teachers, both of which provide some evidence of greater parent engagement in the long term among those who received Validated Participation, even relative to those in the Information Workshop treatment arm. To be clear, these data were collected in a less structured fashion and in much smaller samples as compared with the parent-level data at the heart of our study. Moreover, the data are based on the responses of key informants who were asked to provide aggregated accounts of parental behavior. And yet, our systematic coding of those responses yielded findings that were consistent with our main hypotheses.

Where does that leave us? Ultimately, we conclude that the bulk of our evidence does point to a null finding. But the discrepancies between our open- and closed-ended data leave open several possibilities that need to be addressed in future research. It is possible that treatment effects take longer to become evident than we (and other scholars) routinely consider; and/or that one set of measurement approaches yields less valid responses than we typically assume. Nonetheless, we believe that the notion of ``efficacy traps,'' and the possible role of Validated Participation to enhance efficacy and active citizenship cannot yet be dismissed. Particularly given the relatively low-cost of administering the treatment, and the fact that we detect only positive, and no negative effects, other scholars could incorporate this treatment into related studies of citizen behaviors. We offer ideas and training guides as a supplement to this article should others decide to do so. 

Future research can also help to identify the exact types and dosage of validation that can be delivered most effectively and efficiently. This would require larger-scale studies in terms of numbers of households and locations; the unbundling of treatment arms to better understand the effects of constituent parts; and longer-term studies that would allow for observation of the full vicious/virtuous cycle, including through the causes and consequences of socio-economic status. For those organizations -- including schools, civil groups, and other government and non-government entities -- that seek to increase such active citizen engagement, these findings serve as a reminder that citizens need to believe they are capable of taking those actions and that their efforts will be well received by authorities. This study identifies specific strategies for organizational leaders to create opportunities for targeted citizens to successfully engage and to explicitly recognize those efforts. Our intervention protocol offers several clear, inexpensive, and easily-adapted steps that could be implemented by a wide range of actors involved in social development.

\newpage
\vspace{1cm}
\setstretch{1}
\bibliography{efficacyboost}


\end{document}
